Wednesday, June 30, 2010

Copy Files from multiple subdirectories to 1 folder

Say What?

I have no idea how to condense the title either.  Let me try again...and again...

-Consolidate Files using a batch script
-Copy x number of files at a time into a single folder
-DOS Batch Script to consolidate files from multiple subdirectories
-Cut & paste files from subdirectories into a single directory

Background:
A document server archives 900,000 documents over 6 years, broken down into a subdirectory for each business day.  In order to upgrade the software, I need to copy the files in groups of 5000 files at a time into a single directory for re-processing.  I then have to time it so the previous job finishes processing before the next batch of 5000 records starts.  Sounds simple, right?  WRONG!  I searched the Internet and failed at finding a solution so I had to create my own.

And now, for your processing pleasure....the script.  Remember that all you have to do is create a text file and rename it so it ends with .bat.  This script only copies files ending in .txt.



@echo off & setLocal EnableDELAYedeXpansion
set N=0
FOR /R C:\AndrewTemp\Data %%a in (*.txt) do (
set /a N+=1
if !N! lss 5000 move/y "%%a" c:\AndrewTemp\temp )

Example Walkthrough:
Folder c:\AndrewTemp\Data\1 has 4000 files
Folder c:\AndrewTemp\Data\2 has 4000 files

When the script is run, the files are consolidated into c:\AndrewTemp\temp

After the first pass, c:\AndrewTemp\Data\1 is empty, c:\AndrewTemp\Data\2 has 3000 files, and c:\AndrewTemp\temp has 5000 files.

After the 2nd pass, all 8000 files are in c:\AndrewTemp\temp.

I finished the process by adding the task to Windows Scheduler to run every 70 minutes.


**Note**
The only odd thing about this script is that it still seems to loop through all the files, but stops cut/pasting after 5000 records.  The black DOS window remained on the screen for a good 5 minutes after the files were copied.  I'm sure there is a way to solve, but it wasn't necessary for my application.

No comments:

Post a Comment