Commit Graph

95 Commits

Author SHA1 Message Date
Christian Kamm
6a7b138aa6 Discovery: Query data-fingerprint on root item
Previously the property wasn't queried, meaning the fingerprint logic
couldn't get triggered.
2019-04-12 12:18:02 +02:00
Christian Kamm
440c06c54a Discovery: 403 and 503 on root cause error
Previously these result codes during remote discovery of the sync root
would not cause an error and the discovery would get stuck.

Also extends RemoteDiscovery tests to check for errors on the root item.
2019-04-11 13:48:56 +02:00
Christian Kamm
702a494cbb PropagateIgnore: Default to NormalError for INSTRUCTION_ERROR
Previously if one set the instruction to ERROR while forgetting to set
an error status, it'd propagate as FileIgnored. Now the default is
NormalError for INSTRUCTION_ERROR and FileIgnored for
INSTRUCTION_IGNORE.
2019-04-11 13:48:56 +02:00
Christian Kamm
04268becd1 Vfs: Clear up relationship between _type and pin state
The pin state is a per-item attribute that has an effect on _type:
AlwaysLocal dehydrated files will be marked for hydration and OnlineOnly
hydrated files will be marked for dehydration.

Where exactly this effect materializes depends on how the pin states are
stored. If they're stored in the db (suffix) the dbEntry._type is
changed during the discovery.

If the pin state is stored in the filesystem, the localEntry._type must
be adjusted by the plugin's stat callback.

This patch makes pin states behave more consistently between plugins.
Previously with suffix-vfs pin states only had an effect on new remote
files. Now the effect of pinning or unpinning files or directories is as
documented and similar to other plugins.
2019-04-09 12:46:35 +02:00
Christian Kamm
ca003823d5 Detect missing server data during discovery #7112
This has two positive effects:
- We can put the error on the particular file that has missing data
- We can sync all other files
2019-04-09 10:09:56 +02:00
Christian Kamm
83c30f7dc7 Vfs: Better handling and more tests for suffix file renames
Previously removing the vfs suffix of a file always triggered a
conflict. Now it may just cause a file download.

This was done because users expected symmetry in the rename actions and
renaming foo -> foo.owncloud already triggers the "make the file
virtual" action. Now foo.owncloud -> foo triggers the "download the
contents" action.
2019-04-09 10:09:34 +02:00
Christian Kamm
aca19c7a6d Vfs: Hydrating a virtual is SYNC not NEW #7101
Previously it'd be NEW(ItemTypeFile), but now it has changed to be
SYNC(ItemTypeVirtualFileDownload) which allows better classification.
2019-03-28 17:30:28 +01:00
Christian Kamm
b181740b30 VFS: Unbreak behavior for rename+hydrate #7001
Users can rename a file *and* add/remove the vfs suffix at the same time
leading to very complex sync actions. This patch doesn't add support for
them, but adds tests and makes sure these cases do not cause unintened
behavior.

The rename will be propagated, but the users's hydrate/dehydrate request
will be ignored.
2019-03-21 11:13:16 +01:00
Christian Kamm
3d18e52017 Discovery win: Fix detection of case-only renames
Previously they were detected as DELETE+NEW because if "a" is renamed to
"A" then QFile::exists("a") will still return true on Windows.
2019-02-28 08:43:41 +01:00
Christian Kamm
9b7eba0174 Vfs: Make dehydration a SYNC and not a NEW action
That change will be useful for the notifications. Previously the
dehydrated files were reported as "newly downloaded", now they're
reported as "updated".
2019-02-11 13:35:14 +01:00
Christian Kamm
30d7157eb8 Vfs suffix: Fix dehydration creating the wrong db entry 2019-02-11 13:35:14 +01:00
Christian Kamm
210f309e6e PinStates cleanup
- SyncJournalDB functions now behind internalPinStates() to avoid
accidental usage, when nearly everyone should go through Vfs.
- Rename Vfs::getPinState() to Vfs::pinState()
2019-02-11 13:35:14 +01:00
Christian Kamm
010abe4c82 Pin state updates
- unspecified and inherited are different
- move enum to header in common/
- access through Vfs instead of directly in Journal
2019-02-11 13:35:14 +01:00
Christian Kamm
02f0e18913 Discovery: Files can have dehydrate/download actions
This will be used in conjunction with vfs plugins that detect whether a
file has a pending hydration/dehydration through independent means and
communicate that to the discovery through local file type.
2019-02-11 13:35:14 +01:00
Christian Kamm
e0200fbbd2 Discovery: Add signal for silentlyExcluded files
This allows SyncFileStatusTracker to also know about these. After all
its information is used to provide icons for them too.
2019-02-11 13:35:14 +01:00
Olivier Goffart
dfdcd3ff54 Move: Fix move detection in directory move on the other side 2019-01-22 08:14:12 +01:00
Olivier Goffart
303922b80f Move: add more test and fix move within moves 2019-01-22 08:14:12 +01:00
Christian Kamm
90a8eaa949 vfs suffix: Ignore server files or synced files with the suffix #6953 2019-01-18 11:03:01 +01:00
Christian Kamm
df1723c9b2 vfs: Remove newFilesAreVirtual - use root PinState instead
This unifies how to deal with pin states.

Also enable reading a folders direct pin state vs its effective pin
state.
2019-01-07 10:09:03 +01:00
Olivier Goffart
3313de9066 Discovery: Handle the blacklistFiles from the server capabilities
Issue #434

Ideally one could add the blacklist to the exlucde reggexp, but this
is simpler
2018-12-22 18:19:37 +01:00
Christian Kamm
fac6ad354f vfs: Use PinState in sync algorithm #6815
New files are virtual if the file's pin state is OnlineOnly.
2018-12-19 14:04:24 +01:00
Olivier Goffart
3eabe35520 Discovery: Set right direction when restoring deleted discovery because it has modified files
(Catched by a faillure of t1.pl)
2018-12-19 10:54:48 +01:00
Christian Kamm
18ab3f5dcd vfs: Separate vfs availability from new-files-virtual
This helps support 2.5 settings where there are virtual files in the
tree but new files aren't created virtual.

It's also a prelude for #6815

There's currently no way of
- upgrading vfs plugins (a silent suffix->winvfs upgrade is attempted
  once only, when moving to master)
- disabling vfs capabilities outright
2018-11-26 13:02:48 +01:00
Christian Kamm
4f56366fd5 vfs: Ensure SyncOptions::_vfs is never null
- Create a VfsOff derived class
- Make it a shared pointer shared with Folder::_vfs
2018-11-26 13:02:48 +01:00
Christian Kamm
6cb071da47 Generalize Result<> class, add Optional<>
To make it nicer to use outside of HTTP results.
2018-11-26 13:02:48 +01:00
Christian Kamm
f500539c44 vfs: Make some behaviors suffix-vfs specific 2018-11-26 12:59:51 +01:00
Christian Kamm
f6863964ee vfs: Allow remote renames to propagate as such 2018-11-26 12:59:51 +01:00
Christian Kamm
12e5313d85 vfs: Implement dehydration tagging 2018-11-26 12:59:51 +01:00
Christian Kamm
0c83794227 vfs: Fix suffix detection and handling 2018-11-26 12:59:51 +01:00
Christian Kamm
446a82c00d Fixes after rebase to master
- use vfs suffix in ProcessDirectoryJob
- fix include vfs.h
- fix local vio passing vfs
- fix checksum computation
- vfs mode use
- mingw lambda compile issue
2018-11-26 12:54:55 +01:00
Olivier Goffart
3b386e0b33 csync_vio_locale: use QString for the path in opendir
So we don't have to convert to utf8 and back again
2018-11-15 07:26:44 +01:00
Olivier Goffart
eef3e9086a Exclude: do everything with QString wiuthout converting to char* 2018-11-15 07:26:44 +01:00
Olivier Goffart
f92cfdb3a3 ProcessDirectoryJob::process: optimize so there is only one map 2018-11-15 07:26:44 +01:00
Olivier Goffart
682cc6f199 Sync: optimize by removing setFileRecordMetadata
Inh most case we already have a record from before, so avoid doing a useless
lookup in the database.
In owncloudpropagator.cpp, directories do not have a checksum so no need
to call a function that preserves it
2018-11-15 07:26:44 +01:00
Olivier Goffart
164993ddf8 Workaround GCC < 7 bug.
https://stackoverflow.com/questions/32097759/calling-this-member-function-from-generic-lambda-clang-vs-gcc
2018-11-08 11:22:28 +01:00
Olivier Goffart
69251ed83e Discovery: fix double emission of finished in case of error 2018-11-08 11:22:03 +01:00
Olivier Goffart
dc63996773 Discovery: Fix downloading files when database is used for local discovery
This also fix the currently failling LockedFilesTest
2018-11-07 13:46:54 +01:00
Olivier Goffart
7803ef397e Discovery: Fix renaming on windows
buf.type is ItemFileSkip because csync_vio_local_stat does not set this field
2018-11-02 16:47:56 +01:00
Olivier Goffart
3823aabdbc New Discovery: Fix trailing slash causing failure on windows 2018-11-01 12:21:36 +01:00
Christian Kamm
874211d0e4 Discovery: Virtual file handling adjustments
- adjust virtual file path handing
- helpers for vfs suffix adding/removal
- helpers for isDirectory/isVirtual on SyncJournalRecords
- be clear about what PathTuple _local/_server mean
2018-10-24 10:50:54 +02:00
Olivier Goffart
d43463e603 Database: Add an index on the parent path
So we can quickly query the items in a parent directory

This uses a custom slite3 function, which means that when downgrading the client,
or using another tool to add entries in the database, any insertion in the metadata
table will produce an error: "unknown function: parent_hash()"
(This will crash the client 2.5)
2018-10-23 11:14:01 +02:00
Christian Kamm
16a5754cf8 Discovery: Fix log output for instructions 2018-10-19 11:23:50 +02:00
Christian Kamm
2a5fbd1913 Discovery: Don't rebuild invalidFilname regex each call 2018-10-19 10:51:25 +02:00
Christian Kamm
ff7e9af35c Discovery: Introduce smaller functions 2018-10-19 10:45:11 +02:00
Christian Kamm
d389b37d90 Excludes: drop csyncTraversalMatchFun()
The new discovery can call the traversal match function directly.
2018-10-19 10:28:22 +02:00
Olivier Goffart
2e16f654ea Merge remote-tracking branch 'owncloud/master' into new_discovery_algo
Conflicts:
	src/csync/csync_update.cpp
        test/csync/csync_tests/check_csync_update.cpp
2018-10-18 13:22:29 +02:00
Christian Kamm
e42b0bce4e Discovery: restructure processFileAnalyzeLocalInfo 2018-10-17 11:29:45 +02:00
Christian Kamm
b9f17aeaf1 Discovery: easy conditions first in processFileAnalyzeRemoteInfo
Removing two levels of indent that way
2018-10-17 10:48:15 +02:00
Christian Kamm
43f9d8ac5e Discovery: Add comments 2018-10-17 10:42:58 +02:00
Olivier Goffart
ab124a76d8 Discovery: make sure finished is not called twice, even in case of errors 2018-10-16 16:18:39 +02:00