Robocopy 拖慢部機？

bongbong3481

公司有個sharefolder（大約40G 左右，好多細FILE，EXCEL 同埋OUTLOOK EMAIL FILE 多，FILENAME繁簡英都有），每日系自己部機起個日期FOLDER ，然後用ROBOCOPY 將SHAREFOLDER COPY 去我部機［部機有2個disk，一個裝window及軟件，一個專門backup data］，一行ROBOCOPY 部機就有D 慢，吾係好順。/R/W/MT 已經SET 到好細，都沒咩用。

請問點解決？

s84292

公司有個sharefolder（大約40G 左右，好多細FILE，EXCEL 同埋OUTLOOK EMAIL FILE 多，FILENAME繁簡英都有） ...
bongbong3481 發表於 2023-11-24 15:13

ROBOCOPY你預實慢架
佢OS 核心同IO都要大負載

一係你 limit ROBOCOPY 個bandwidth
在ROBOCOPY 之後加 /ipg:750
750呢個數字可以自己改,試到你岩為止
可以試唔落/MT, 由得WINDOWS自己行為預設數(應該為8)

一係就改用DFS replication

s20012797

Based on your description, it seems like the issue is related to the size of the files being copied and the number of files being copied. Here are some possible solutions that you can try:

Increase the number of threads: You can try increasing the number of threads used by Robocopy by adding the /MT option followed by the number of threads you want to use. For example, /MT:16 will use 16 threads. This can help speed up the copying process by allowing Robocopy to copy multiple files simultaneously .

Disable logging: Robocopy logs every file that it copies, which can slow down the copying process. You can disable logging by adding the /NFL and /NDL options to your command. For example, /NFL /NDL will disable logging of file and directory names .

Use unbuffered I/O: You can try using unbuffered I/O by adding the /J option to your command. This can help speed up the copying process for large files .

Disable Windows Copy Offload: You can try disabling Windows Copy Offload by adding the /NOOFFLOAD option to your command. This can help speed up the copying process for large files .


it is possible to use Python scripts as an alternative to solve the problem. Python provides various libraries and modules that can help with file operations and copying files.

One option is to use the `shutil` module in Python, which provides a high-level interface for file operations. You can use the `shutil.copytree()` function to recursively copy a directory and its contents to another location. Here's an example:

python:

1. import shutil
   
2. 
   
3. source_folder = 'path/to/source/folder'
   
4. destination_folder = 'path/to/destination/folder'
   
5. 
   
6. shutil.copytree(source_folder, destination_folder)

複製代碼

This will copy the entire directory tree from the source folder to the destination folder.

You can also use the `os` module in Python to perform file operations. The `os.listdir()` function can be used to get a list of files and directories in a given folder, and then you can use the `shutil.copy()` function to copy individual files. Here's an example:

python:

1. import os
   
2. import shutil
   
3. 
   
4. source_folder = 'path/to/source/folder'
   
5. destination_folder = 'path/to/destination/folder'
   
6. 
   
7. for file_name in os.listdir(source_folder):
   
8.     source_file = os.path.join(source_folder, file_name)
   
9.     destination_file = os.path.join(destination_folder, file_name)
   
10.     shutil.copy(source_file, destination_file)

複製代碼

This will copy each file from the source folder to the destination folder.

Using Python scripts gives you more flexibility and control over the copying process. You can customize the script according to your specific requirements and handle any additional operations or conditions as needed.

Remember to adjust the file paths (`source_folder` and `destination_folder`) in the examples to match your specific setup.

bongbong3481

回覆 3# s20012797

我臨時用左python copy tree 頂住先，但robocopy 有太多功能選項玩，prefer robocopy （我估用python 都模擬到，但要寫好多code ,自己懶，沒辦法）

tgo

你部機乜嘢config? 我都有咁用，完全冇問題。你copy緊開埋個task manager睇埋個cpu/hdd 扯到幾高。

公司有個sharefolder（大約40G 左右，好多細FILE，EXCEL 同埋OUTLOOK EMAIL FILE 多，FILENAME繁簡英都有） ...
bongbong3481 發表於 2023-11-24 15:13

java2

有D 慢即係點慢? 日日 copy 40G files 去你部機HDD?
為乜? 如果係Backup 應用incremental copy

公司有個sharefolder（大約40G 左右，好多細FILE，EXCEL 同埋OUTLOOK EMAIL FILE 多，FILENAME繁簡英都有） ...
bongbong3481 發表於 2023-11-24 15:13

bongbong3481

你部機乜嘢config? 我都有咁用，完全冇問題。你copy緊開埋個task manager睇埋個cpu/hdd 扯到幾高。


...
tgo 發表於 2023-11-25 07:13

8核CPU，16G 行win10，用TASK MANAGER 睇，
CPU 扯得吾高，得幾個%，反而HARD DISK 去到100%

bongbong3481

回覆 6# java2

其實最快是用incremental copy， 不過有d file 改完，想還原番去前幾個版本就有問題。所以每日起個date folder 將成個sharefolder backup

java2

用你自己部機HDD 做backup?  日日寫40G? 慢都正常, HDD I/O 喎. 你唔覺呢個方法本身就有問題?

回覆  java2

其實最快是用incremental copy， 不過有d file 改完，想還原番去前幾個版本就有問題。所以每 ...
bongbong3481 發表於 2023-11-25 11:15

bongbong3481

回覆 9# java2

其實copy 速度慢都沒所謂，最大問題系令到成部機都好慢，影響做野。 我部機有2個disk，一個裝window及軟件，另一個backup data，已經分開disk。
（其實我都有用7zip 黎backup 個sharedrive，速度都好快，不過吾知咩原因，成日話有d file 搵吾到，所以成個folder 去copy。  另外，夜晚又要backup outlook archive，沒辦法夜晚做 ）