Fix I/O error when creating empty file

[PythonNut]

httplib 0.9.2 raises an exception when determining the progress of a zero-length transaction, which causes an I/O error. This skips the check entirely, since the result isn't used anyway when the total_size is zero.

I don't know how many people are affected by this, but obviously most files are empty when created. (Is it possible to create a nonempty file in one operation?)

Anyway, with this patch GDriveFS seems quite usable!

[dsoprea]

We can't merge this until we figure out why you're the only one experiencing this problem.

Yes, it's possible to create an empty file. Any filesystem would have to support this. The only alternative is forcing you to pass data.

[PythonNut]

Yes, it's possible to create an empty file. Any filesystem would have to support this. The only alternative is forcing you to pass data.

I sort of assumed that. I'm no expert, so thanks for clearing this up.

We can't merge this until we figure out why you're the only one experiencing this problem.

No idea why. The only thing that's abnormal for me is that I'm running this on ARM. Everything else is stock Arch Linux.

Is there any specific information you'd like to know? I could get you the traceback, for instance.

[dsoprea]

I have three immediate impressions:

1. I have no idea why, other then to lazily speculate at a race condition inducing an issue that doesn't come up in anyone else's environments (at least the outspoken users).
2. Start injecting debug statements to get a better idea of the shape of the issue.
3. If you can deterministically reproduce it, try to do so in the Ubuntu control environment: There is a Vagrantfile included with the project (vagrant/). That would allow us to be sufficiently in-sync with our environments.

[PythonNut]

Start injecting debug statements to get a better idea of the shape of the issue.

I have a fairly good idea of what the issue looks like now. When MediaDownloadProgress.total_size = 0, MediaDownloadProgress.progress() causes a ZeroDivisionError:

Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/gdrivefs/gdfs/gdfuse.py", line 358, in create
mime_type)
File "/usr/lib/python2.7/site-packages/gdrivefs/gdfs/opened_file.py", line 224, in __init__
self.__load_base_from_remote()
File "/usr/lib/python2.7/site-packages/gdrivefs/gdfs/opened_file.py", line 311, in __load_base_from_remote
self.mime_type)
File "/usr/lib/python2.7/site-packages/gdrivefs/gdtool/drive.py", line 55, in wrapper
return f(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/gdrivefs/gdtool/drive.py", line 461, in download_to_local
p = status.progress()
File "/usr/lib/python2.7/site-packages/googleapiclient/http.py", line 153, in progress
return float(self.resumable_progress) / float(self.total_size)
ZeroDivisionError: float division by zero

We create the MediaDownloadProgress object with a zero total_size field here. And, unfortunately, that's the end of my tracing, because the value we feed it is directly tied to the HTTP response from Google, as we learn from this line that the chunk size really is 0.

 2016-05-06 17:11:12,425 [gdrivefs.gdtool.chunked_download DEBUG] Checking if done. PROGRESS=(0) TOTAL-SIZE=(0)

I tried to reproduce this on a Debian system which has largely nothing to do with my Arch System (sorry, I only have arm Linux devices, and there's no Vagrant for arm) and the ZeroDivisionError is still generated. In addition, both systems produce:

2016-05-06 17:24:20,950 [gdrivefs.gdfs.fsutility ERROR] FUSE error [FuseOSError] (5) will be forwarded back to GDFS from [create]: [Errno 5] Input/output error
2016-05-06 17:24:20,953 [gdrivefs.gdfs.fsutility DEBUG] <<<<<<<<<< create(8) (30153)
   unique: 36, error: -5 (Input/output error), outsize: 16

At this point I should note that the I/O error does not actually prevent the file from being created, and the file is later updated by the change detector:

2016-05-06 17:23:37,714 [gdrivefs.gdtool.drive DEBUG] We received (1) changes 

(Please pay no attention to the dates and times, these were produced out of order over multiple tests)

On the Debian system, the I/O error doesn't seem to be forwarded, and because the operation actually took place, everything appears to work correctly from the perspective of the user. On the Arch system, the I/O error is forwarded and causes the program that requested the operation to think it has failed.

[amurashkin17]

I had the same problem. status.progress() fails if the downloaded size is 0. It always happens for new target files or if googledrive target file has size 0.

This commit has resolved it.

BTW Why does gdrivefs try to download anyway, the target files are usually opened with O_TRUNC. So what is the point in downloading?

File "/usr/lib/python2.7/site-packages/gdrivefs/gdtool/drive.py", line 461, in download_to_local
p = status.progress()
File "build/bdist.linux-x86_64/egg/apiclient/http.py", line 108, in progress
return float(self.resumable_progress) / float(self.total_size)
ZeroDivisionError: float division by zero

See gdfs_log.txt file for complete log.

[PythonNut]

@dsoprea any thoughts on this?

[PythonNut]

@dsoprea am I correct in reading that you've fixed the issue yourself?

[dsoprea]

Yes.

On Sun, Jun 26, 2016 at 8:42 PM, PythonNut [email protected] wrote:

@dsoprea https://github.com/dsoprea am I correct in reading that you've
fixed the issue yourself?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#157 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AArramPUvBqiuIQp68EmozfPZgLrY2oMks5qPxxkgaJpZM4IYmmV
.