Thursday, June 24, 2021

Monitoring the real progress in Linux while copying file to a slow USB drive

For tracking file copy process in Linux systems, one option is using Pipe Viewer (pv). It is a terminal-based tool for monitoring the progress of data through a pipeline. 

$ pv source-file > destionation-file
1,36GiB 0:05:24 [4,28MiB/s] [==================================================>] 100% 

But when it comes to copy files to slow USB drives, the progress may reach to 100% immediately and then stay hanged without giving any clue about when the copying will be finished. This is because of that the Linux kernel first makes disk writes into cache, and later flushes them to disk asynchronously. To get a real idea during the process about when it will completed, we may set this cache to a specific size (10 MB was enough in our case) like below:

$ sudo sysctl vm.dirty_bytes=104857600

For more information: