Commit Graph

123 Commits (master)

Author SHA1 Message Date
Nick Craig-Wood e3c11c9ca1 mount: add --async-read flag to disable asynchronous reads
See: https://forum.rclone.org/t/constantly-high-iowait-add-log/14156
2020-02-27 16:12:33 +00:00
Nick Craig-Wood 17b4058ee9 mount: constrain to go1.13 or above otherwise bazil.org/fuse fails to compile 2020-02-26 21:26:46 +00:00
Nick Craig-Wood 9663f9b2ab mount: ignore --allow-root flag with a warning as it has been removed upstream
For background see: https://github.com/bazil/fuse/issues/144
2020-02-26 21:11:25 +00:00
Nick Craig-Wood ec1271818f mount2: hide mount2 command for the moment 2020-02-11 14:28:13 +00:00
Nick Craig-Wood 706da80d88 mount: don't build on go1.10 as bazil/fuse no longer supports it 2020-01-08 08:44:02 +00:00
Nick Craig-Wood c21a4fee58 mount,cmount: make sure we call unmount when exiting 2019-11-11 22:08:52 +00:00
Nick Craig-Wood 21c6babdbb mount: enable async reads for a 20% speedup
Now that the vfs can cope with out of order reads we can enable the
async read feature for an increase in througput on the local disk of
about 20%.
2019-11-11 16:20:27 +00:00
Brett Dutro 378a3f4133 mount: replace use of WriteAt with Write for cache mode >= writes and O_APPEND
os.File.WriteAt returns an error if a file was opened with O_APPEND.
This replaces it with os.File.Write if the file was opened with
O_APPEND.
2019-10-26 17:27:52 +01:00
Nick Craig-Wood 59026c4761 mount, cmount: don't pass huge filenames (>4k) to FUSE as it can't cope 2019-10-21 20:51:11 +01:00
Nick Craig-Wood 15b1feea9d mount: fix panic on File.Open - Fixes #3595
This problem was introduced in "mount: allow files of unkown size to
be read properly" 0baafb158f by failure to check that the
DirEntry was nil or not.
2019-10-06 19:26:58 +01:00
Nick Craig-Wood 08b86cc94b mount: skip tests on <= 2 CPUs to avoid lockup in #3154 2019-09-20 11:53:08 +01:00
Nick Craig-Wood 0baafb158f mount: allow files of unkown size to be read properly
Before this change, files of unknown size (eg Google Docs) would
appear in file listings with 0 size and would only allow 0 bytes to be
read.

This change sets the direct_io flag in the FUSE return which bypasses
the cache for these files.  This means that they can be read properly.

This is compatible with some, but not all applications.
2019-09-14 13:22:33 +01:00
Nick Craig-Wood ffa1dac10b build: apply gofmt from go1.13 to change case of number literals 2019-09-05 13:59:06 +01:00
Nick Craig-Wood 341d880027 mount: remove nonseekable flag from write files - fixes #3461
Before this change rclone marked files opened for write without VFS
cache with the non seekable flag.

This caused problems with rclone mount layerd with mergerfs.

This change removes the hint and lets rclone do all the checking for
seekability.
2019-08-22 13:13:59 +01:00
Patrick Wang badc8b3293 mount: Fix typo in argument checking 2019-08-20 17:46:04 +01:00
Nick Craig-Wood 57d5de6fba build: fix up package paths after repo move
git grep -l github.com/ncw/rclone | xargs -d'\n' perl -i~ -lpe 's|github.com/ncw/rclone|github.com/rclone/rclone|g'
goimports -w `find . -name \*.go`
2019-07-28 18:47:38 +01:00
Nick Craig-Wood 8beab1aaf2 build: more pre go1.8 workarounds removed 2019-05-08 15:14:51 +01:00
Fabian Möller 48c09608ea
fix spelling 2019-04-30 14:12:18 +02:00
Nick Craig-Wood d966cef14c build: fix problems found with unconvert 2019-02-02 11:45:07 +00:00
Nick Craig-Wood f8ced557e3 mount: print more things in seek_speed test 2019-02-02 11:30:49 +00:00
Nick Craig-Wood 174ca22936 mount,cmount: clip the number of blocks to 2^32-1 on macOS
OSX FUSE only supports 32 bit number of blocks which means that block
counts have been wrapping.  This causes f_bavail to be 0 which in turn
causes problems with programs like borg backup.

Fixes #2356
2018-08-26 17:32:59 +01:00
Nick Craig-Wood 77806494c8 mount,cmount: adapt to sdnotify API change 2018-08-20 12:34:29 +01:00
Oleg Kovalov 06c9f76cd2 all: fix go-critic linter suggestions 2018-08-06 21:14:03 +01:00
Nick Craig-Wood 473e3c3eb8 mount/cmount: implement --daemon-timeout flag for OSXFUSE
By default the timeout is 60s which isn't long enough for long
transactions.  The symptoms are rclone just quitting for no reason.
Supplying the --daemon-timeout flag fixes this causing the kernel to
wait longer for rclone.
2018-07-19 13:26:51 +01:00
Fabian Möller d022c81d99 mount: ensure atexit gets run on interrupt
When running `rclone mount`, there were 2 signal handlers for `os.Interrupt`.

Those handlers would run concurrently and in some cases cause either unmount or `atexit.Run()` being skipped.

In addition `atexit.Run()` will get called in `resolveExitCode` to ensure cleanup on errors.
2018-05-12 10:40:44 +01:00
Nick Craig-Wood 9d8d7ae1f0 mount,cmount: make --noappledouble --noapplexattr and change defaults #2287
Before this change we would unconditionally set the OSXFUSE options
noappledouble and noapplexattr.

However the noapplexattr options caused problems with copies in the
Finder.

Now the default for noapplexattr is false so we don't add the option
by default and the user can override the defaults using the
--noappledouble and --noapplexattr flags.
2018-05-07 20:37:09 +01:00
Nick Craig-Wood 6ce32e4661 mount,cmount: Add --volname flag and remove special chars from it #2287
Before this change rclone would set the volume name from the
remote:path normally.  However this has `:` and `/` in which make it
difficult to use in macOS.

Now rclone will remove the special characters and replace them with
spaces.  It also allows the volume name to be set with the --volname
flag.
2018-05-07 20:37:09 +01:00
Nick Craig-Wood 1755ffd1f3 mount: make Get/List/Set/Remove xattr return ENOSYS #2287
By default bazil fuse will return ENOTSUPP for these.  However if we
return ENOSYS then OSXFUSE (at least) will never call them again
saving round trips though fuse.
2018-05-07 20:37:09 +01:00
Nick Craig-Wood 2b855751fc vfs,mount,cmount: use About to return the correct disk total/used/free
Disks total, used, free now shows correctly for mount and cmount (eg
`df` for Unix or in the Windows explorer).
2018-04-18 18:27:34 +01:00
Nick Craig-Wood 06e3fa3aba mounttest: reduce duplicated code and improve test output #2154
The written out list of tests was replaced with a nested test for
mount and cmount. The tests for each VFS cache mode were also replaced
with nested tests which makes the output and the code much cleaner.
2018-04-08 15:04:14 +01:00
Nick Craig-Wood e5be471ce0 Use io.SeekStart/End/Current constants now for go1.7+ #2154 2018-04-07 11:42:36 +01:00
Nick Craig-Wood 80588a5a6b Replace "golang.org/x/net/context" with "context" for go1.7+ #2154 2018-04-07 11:42:08 +01:00
Nick Craig-Wood 34c45a7c04 mount, cmount: remove addition of O_CREATE to flags on file open #2141
Previously this was adding it in to all file opens which was causing
inefficiencies under Windows where it stats the file using
open/fstat/close.

This change will make stat operations run much quicker under Windows
as they won't have to open the underlying file

This problem was introduced in61b6159a05336bd7ba105766de2d2ff171f7fb81
where we added O_CREATE to all file opens and creates.
2018-03-15 20:48:56 +00:00
Nick Craig-Wood c1a3e363a6 mount: return ENOSYS rather than EIO on attempted link
This fixes FileZilla accessing an rclone mount served over sftp.

See: https://forum.rclone.org/t/moving-files-on-rclone-mount-with-filezilla/5029
2018-03-14 21:10:20 +00:00
Nick Craig-Wood fc32fee4ad mount, cmount: add --attr-timeout to control attribute caching in kernel
This flag allows the attribute caching in the kernel to be controlled.
The default is 0s - no caching - which is recommended for filesystems
which can change outside the control of the kernel.

Previously this was at the default meaning it was 60s for mount and 1s
for cmount.  This showed strange effects when files changed on the
remote not via the kernel.  For instance Caddy would serve corrupted
files for a while when serving from an rclone mount when a file
changed on the remote.
2018-03-04 11:20:22 +00:00
Nick Craig-Wood 54deb01f00 vfs: Make OpenFile and friends return EINVAL if O_RDONLY and O_TRUNC
Before this change Open("name", os.O_RDONLY|os.O_TRUNC) would have
truncated the file.  This is what Linux does, but is counterintuitive.
POSIX states this is undefined, so return an error in this case
instead.  This preserves the invariant O_RDONLY => file is not
changed.
2018-02-26 17:04:27 +00:00
Nick Craig-Wood c277a4096c mount: don't set modtime twice #2021 2018-01-29 20:49:13 +00:00
Nick Craig-Wood a80d8a21dc vfs: add flags parameter to Dir.Create 2018-01-19 13:18:40 +00:00
Nick Craig-Wood 11da2a6c9b Break the fs package up into smaller parts.
The purpose of this is to make it easier to maintain and eventually to
allow the rclone backends to be re-used in other projects without
having to use the rclone configuration system.

The new code layout is documented in CONTRIBUTING.
2018-01-15 17:51:14 +00:00
Nick Craig-Wood dc1c679c65 mount: support truncate properly 2017-11-20 19:42:35 +00:00
Nick Craig-Wood b4083b4371 vfs: rename Fsync to Sync and implement Sync on Node and Handle 2017-11-20 12:46:23 +00:00
Nick Craig-Wood 61b6159a05 mount, cmount: add O_CREATE to Open calls since fuse doesn't seem to supply it 2017-11-20 12:46:22 +00:00
Nick Craig-Wood c560017934 vfs: add Path method to Node and use it to stop reading nil DirEntry
All DirEntry calls now have been checked for nil or converted to use Path.
2017-11-20 12:46:22 +00:00
Nick Craig-Wood 57a5c67729 mounttest: run the tests for all 4 VFS cache modes 2017-11-20 12:46:21 +00:00
Nick Craig-Wood 8951875c21 vfs,mount,cmount,mountlib: allow flags to be overriden by environment variables 2017-11-20 12:46:21 +00:00
Nick Craig-Wood 3fb1e96988 vfs: factor Open logic from Dir.Create into vfs.OpenFile 2017-11-20 12:00:32 +00:00
Fabian Möller d8b3bf014d mount: use sdnotify to signal systemd the mount is ready
When the NOTIFY_SOCKET environment variable is set notify systemd after
the mount is ready.
2017-11-20 11:03:10 +00:00
Nick Craig-Wood 2083ac6e2a vfs: add ECLOSED and tidy errors 2017-11-04 10:24:10 +00:00
Nick Craig-Wood 5634659ea3 mount,vfs: unify Read and Write handles in preparation for ReadWrite handles 2017-11-04 10:24:10 +00:00
Nick Craig-Wood e18122e88b vfs: add tests and subsequent fixes
* Tests for VFS layer
  * Small fixes found during testing
  * Fix Close, Flush and Release behaviour for ReadFileHandle and WriteFileHandle
  * Fix nil object bugs on File
2017-11-04 10:24:10 +00:00