-
-
Notifications
You must be signed in to change notification settings - Fork 870
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bug: Cannot upload-sync big files (>20 mb) #2648
Comments
I have the same problem. |
Many thanks for the feedback re the client and doc's, however this issue is 100% not a bug with the client and it will be 100% something environmental to your environment / server you are using to upload from. Firstly the 100% CPU aspect, you will need to track down what is causing this. Please read https://github.com/abraunegg/onedrive/wiki/Generate-system-performance-data-for-performance-related-issues on how to achieve this. You will need to manually compile the client to enable all the required debug pieces, so that if the client is the cause of the 100% CPU load this can be fixed - but if it is not, then you need to look elsewhere and/or investing in better hardware. In terms of large file upload, I regularly test with files >20Mb - usually in the 100Mb+ range, and use several Linux ISO files in the GB range as well. I never have an upload issue, and never ever have the client hitting 100% CPU load:
Current Master v2.4.25-13-g1a88d33
Current v2.5.0 'alpha-5' PR build:
It potentially also sounds to me like the connection to OneDrive is silently being closed in your environment. How long is the 'upload process' taking on the large file before it fails? If this is taking > 1hr (3600 seconds) , consider changing the If this is failing well before 3600 seconds, you need to look at your network path to Microsoft OneDrive and start your investigations there as to why something in that path is potentially closing the connection. To do this, you will need to look at performing HTTPS Debug Logging - please read: https://github.com/abraunegg/onedrive/wiki/Generate-https-debug-log-for-support The next suggestion here is to drop from HTTP2 to using HTTP1.1 for all operations. Please read the last section of https://github.com/abraunegg/onedrive/blob/master/docs/USAGE.md#curl-compatibility to understand how to change this for your client operations - something in your network path to Microsoft OneDrive may not be entirely compatible with HTTP2. The last suggestion here is, given you will need to manually compile the 'current master' to generate the performance profile information (to validate what is causing your 100% CPU - if coming from the 'onedrive' client), it would be then worth you testing 'alpha-5' in it's current state. Please read:
The code in 'alpha-5' has been 100% re-written , has many fixes and changes (this is alpha code, but has the maturity of 12 months already). The point here is - any bug in v2.4.x will not be fixed. If the issue is seen in 'alpha-5' then it can be looked at. Regardless of this, at the moment it all points to environmental factors | network factors unique to your environment and not a bug with this client - sorry. As such, this bug is not a bug that can be replicated (see my evidence above) and it is 100% most likely something environmental to your setup which you need to diagnose further. I am happy to communicate via email with the applicable debug logs / performance data to try and assist you diagnose this further. |
wow... never estimated such quick and long and helpful response... I whish microsoft would one day be able to be only 10% as fast and accurate on problems... THANK YOU SO MUCH for that wonderful piece of software and your afford! |
This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
Describe the bug
Hi!
First I have to say how impressed I am by the onedrive linux client... installation is perfectly described (I am on debian 12), lots of config options and perfect for my (rather simple) use case...
I want to use onedrive for my vps server backup. System runs on debian 12.
I create the backup using rsync into a local directory, zip that directory with max compression, encrypt the file using openssl and move it to my onedrive sync directory.
First run of the sync process took nearly forever although I only wanted to sync one single (nearly empty) directory (with 330k files in onedrive on other folders)... I found somewhere that I should avoid the skip_list file, that did the trick...
My problem is that the process simply stops working (at 100% cpu load) when I try to upload the 800mb file in one piece...
It works great if I split the files to 15-18 mb but even 20 mb caused the client to stop working...
Running this command:
onedrive --synchronize --single-directory "BIC/VPSServer" --verbose --verbose
And nothing else happens...
I killed the process and wanted to check the sync status:
onedrive --display-sync-status
==> onedrive synced the rest of the file...
This also works for the complete 800mb file... If start "normal" sync the process stops at 1%... I kill the process and run the display-sync-status and the file gets uploaded...
Any ideas? Would be nice to be able to use the sync thing in my backup script without creating 15mb chunks of the file...
Thanks in advance!
Operating System Details
Client Installation Method
From Distribution Package
OneDrive Account Type
Personal
What is your OneDrive Application Version
v2.4.25-1+np1
What is your OneDrive Application Configuration
What is your 'curl' version
Where is your 'sync_dir' located
Local
What are all your system 'mount points'
/dev/vda1 on / type ext4 (rw,relatime,discard,errors=remount-ro)
What are all your local file system partition types
How do you use 'onedrive'
no monitoring, sync of a single folder between vps linux server and windows client in both directions... Ignoring all other folders / files.
Steps to reproduce the behaviour
Sync a new file created in linux with >20mb fails.
Smaller files work without problem. (see above)
Complete Verbose Log Output
Screenshots
No response
Other Log Information or Details
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: