UNKNOWN IO ERROR during download on some devices

[cemrich]

Zapp-Version: 3.5.0

Android-Version: 10

Devices:

• Oneplus

Could be related to timeout settings in Fetch.

[alexanderadam]

I guess it's better to answer your question (which version of Android are you using?) here:

I'm using Android 10 on all of my devices (Samsung S9 and Samsung Galaxy Tab S6).

Furthermore I can confirm that this issue still exists:

I used this video to test it.

Also I don't think that's a (client side) bandwidth issue. Wifi strength is perfect, too:

But it still just happens only on videos with bigger file size.
I'm also curious why it's not causing some kind of timeout error but an unknown error instead.

[cemrich]

I'm also curious why it's not causing some kind of timeout error but an unknown error instead
Yes, that is interesting. I think this might be an Android 10 issue because on user reported this error on Android 10 but not on Android 9 using the same wifi network.

I will try to reproduce it and maybe come back to you.

[cemrich]

Curious. I can't even start a download of this show on my device.

[cemrich]

• Android 10, fresh app install, large file: download gets queued but never proceeds (update: proceeds very, very slow)
• Android 10, fresh app install, small file: download runs just fine
• Android 7, fresh app install, large file: download runs just fine
• Android 7, fresh app install, small file: download runs just fine

I think I narrowed this error down to large file sizes on Android 10 devices. Maybe this is a fetch problem of some kind?

[cemrich]

It is not related with the MediaStore framework.

[cemrich]

The sd card is causing this issue.

@alexanderadam I don't know if you have enough storage space left on your device; But does the same error occur if you do not download to sd card but to internal storage? On my device I cannot download any large file to sd card. It does not even show a notification. Downloading to internal storage seems to work fine. Can you verify this?

[alexanderadam]

Interesting thought, Sherlock! 🕵️‍♀️

I cannot check the file size at the moment but it could be the 4 GB limit of FAT32 filesystem that appears here.
Furthermore I cannot check whether it is working by downloading on internal storage at the moment but I'm coming back on you once I checked it.

UPDATE: @cemrich sorry, this issue also happens on internal space in my case. 😞 I still believe that the 4 GB limit could happen, though. The 4 GB-bug of FAT32 cannot be the cause in this case though, because the file only has a size of 828 M.

If I try to download the HQ video on my working machine (a desktop computer), the speed varies drastically! So it could be that the server simply delivers it slowly sometimes (or speed depends where you are) and we're therefore really running into a timeout.

This is the same machine and connection where I made the speedtest (> 120 Mbit/s in each direction) before and it now has speeds of 56KB/s when downloading the file.
Yes, you read that right: KB! This is taking ages.

It even wen't down to 23KB/s for some moments. Thus I'm back to the theory that it might be releated to a timeout issue.
I have to admit that it wen't up to three-figures speeds again afterwards but I can imagine that these flaky speeds might cause running into tight timeouts.

If this command is much faster for you, you obvious are simply getting a faster delivery than me (could be depending on the country / akamai zone):

$ wget http://wdrmedien-a.akamaihd.net/medp/ondemand/weltweit/fsk0/210/2104880/2104880_26030869.mp4

Would it be possible to make the timeout configurable?
And / or can you create a build that saves a backtrace/debug data to a file when this issue happens?
This could probably help you having a look into it.

[cemrich]

It is most definitely caused by the sd card. My Android 7 test device is having issues too - but only when downloading to sd card.

[cemrich]

The 4 GB-bug of FAT32 cannot be the cause in this case though

No, the files I tested were much smaller.

sorry, this issue also happens on internal space in my case

Do you think it is the same issue? I wonder, if we are debugging two different issues here.

If this command is much faster for you, you obvious are simply getting a faster delivery than me

It averages around 1.5MB/s but is changing very fast.

I think I can improve timeouts, set another downloader for Fetch to use and release a new beta. This does not fix my sd card issue but maybe your timeout / io error...

[alexanderadam]

Do you think it is the same issue? I wonder, if we are debugging two different issues here.

I have the feeling that UNKNOWN_IO_ERROR is simply not a very speaking error class. It could probably be any unhandled error. Funnily enough, people opened unhelpful issues for this, too. 😉
But maybe some of the existing issues is similar to your configuration?

I think I can improve timeouts, set another downloader for Fetch to use and release a new beta.

Sounds good. I will give you feedback again, after I was able to test it.

Is this comment helping in any way?

You can enable logging in Fetch:

 final FetchConfiguration fetchConfiguration = new FetchConfiguration.Builder(this)
                .enableLogging(true)
                .setDownloadConcurrentLimit(3)
                .build();

I still believe that the 4 GB limit could happen, though. The 4 GB-bug of FAT32 cannot be the cause in this case though, because the file only has a size of 828 M.

I just found something regarding this (other) issue:

On enqueue Fetch will try to pre allocate the file on local storage. If it fails an error is returned FILE_ALLOCATION_FAILED. This feature can be turned off in the fetch configuration. It is on by default.

[cemrich]

Is this comment helping in any way?

Yes, I enabled logging, but nothing helpful there. I tweaked the parameters quite a bit. It has to be better now ;)

I just found something regarding this (other) issue:

Did you ever see a FILE_ALLOCATION_FAILED error?

[alexanderadam]

Did you ever see a FILE_ALLOCATION_FAILED error?

No, I need a file bigger than 4 GB to provoke that issue and I don't know any video with that size (yet).

[cemrich]

The beta version will have to wait a bit. OkHTTP causes other issues (like errors are not recognized correctly). I will have to test different options before releasing anything.

[alexanderadam]

I know that I wrote this before already but I really hope that you know that people value the work you're doing. Thank you so much for doing this.

[cemrich]

I just released 3.5.1-beta.1 with increased timeouts and OkHTTP downloader. Additionally files are no longer pre allocated which fixes my sd card problem. Please give it a try and report any errors!

[ybrhue]

Sadly doesn't fix the issue for me.
Can't download neither internal need r external, switched back to 3.4.

Thanks a lot tough for your awesome work!
Android 10 on a rooted redmi note 8 pro (xiaomi).

[alexanderadam]

I can confirm that this exact bug is still present in 3.5.1-beta.
Is there any way I could give you a backtrace or any other helpful data?

[cemrich]

@ybrhue Do you see a UNKNOWN_IO_ERROR inside the error notification? Does the download fail immediately or after some download progress? Does it occur only for larger files or for smaller ones too?

[cemrich]

I just published another release 3.5.1-beta.2 to further test this issue. There is now a small "Report"-Button inside the download error notification which will give you the opportunity to send me a detailed error report.

Feel free to add further information to the mail or to paste the error stacktrace right here into this issue.

Thanks for your help!

[alexanderadam]

Does the download fail immediately or after some download progress? Does it occur only for larger files or for smaller ones too?

After some download progress and only on larger files.

There is now a small "Report"-Button inside the download error notification which will give you the opportunity to send me a detailed error report.

I don't have an error notification in this release and I can't see a Report button.

[cemrich]

I don't have an error notification in this release and I can't see a Report button.

@alexanderadam There should be one. Can you please confirm there is no notification by killing the connection during download?
This bug is getting weirder and weirder...

[alexanderadam]

Sorry, my fault: I had to swipe the notification down again to see the sharing thingy and this wasn't really obvious to me at first because the UI didn't gave me a proper hint to do so.

However, it seems that this will only open the email program with your email address as receiver. It is not attaching a backtrace or any other info, though.
Can you please check again whether there should be a stacktrace/debug data be attached?

[cemrich]

However, it seems that this will only open the email program with your email address as receiver. It is not attaching a backtrace or any other info, though.

There should be a bunch of text inside the mail. I confirmed it works at least with K9 mail, but this should be pretty standard with all other mail apps. Maybe you mail app cannot process the intent used by ACRA crash reporting...

[cemrich]

I got the same error message from a few people using Android 10 devices downloading to internal memory:

java.io.IOException: write failed: EBADF (Bad file descriptor)
        at libcore.io.IoBridge.write(IoBridge.java:544)
        at java.io.FileOutputStream.write(FileOutputStream.java:392)
        at com.tonyodev.fetch2core.x$a.g(StorageResolverHelper.kt:1)
        at com.tonyodev.fetch2.v.f.k(SequentialFileDownloaderImpl.kt:7)
        at com.tonyodev.fetch2.v.f.run(SequentialFileDownloaderImpl.kt:46)
        at com.tonyodev.fetch2.v.c$a.run(DownloadManagerImpl.kt:10)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.lang.Thread.run(Thread.java:919)
        Caused by: android.system.ErrnoException: write failed: EBADF (Bad file descriptor)
        at libcore.io.Linux.writeBytes(Native Method)
        at libcore.io.Linux.write(Linux.java:294)
        at libcore.io.ForwardingOs.write(ForwardingOs.java:241)
        at libcore.io.BlockGuardOs.write(BlockGuardOs.java:416)
        at libcore.io.ForwardingOs.write(ForwardingOs.java:241)
        at libcore.io.IoBridge.write(IoBridge.java:539)

I got the error on my test device but I have no clue on how to reproduce it...

[cemrich]

The error may be related to tonyofrancis/Fetch#454

[cemrich]

A new version 3.6.0 is out and will hit F-Droid in a couple of days. Can you please test if downloading behaves differently now? I may have fixed the error but I am not really sure...

[alexanderadam]

I will test it. I hope I won't miss the release (you can remind me in case I forget it).

[cemrich]

@alexanderadam It is released!

[alexanderadam]

@cemrich seems to work perfectly! 🎉

Just a small question: in case a download can't be finished there's still the half downloaded video file around somewhere, right?
Can this be deleted as well? I cannot test it anymore since the downloads are working fine now 😉

I guess this issue can be closed then? And if somebody else still has this issue, this could be reopened anyway later on.

PS: I know that I wrote this already but I really like the UI changes you did by implementing this. This is really great work. 👍

[cemrich]

seems to work perfectly! 🎉

Yay! Finally!

Just a small question: in case a download can't be finished there's still the half downloaded video file around somewhere, right?
Can this be deleted as well?

No, they don't get deleted, but maybe should... I created issue #212
Cancelling a download however does delete it.