Commit Graph

338 Commits

Author SHA1 Message Date
Markus Goetz
e81400c090 Reconcile: Sort already during discovery #7445 2019-09-09 11:39:44 +02:00
Christian Kamm
99fd45051a Discovery: If a move is forbidden, restore the source
Previously the source was deleted (or attempted to be deleted), even if
the new location was not acceptable for upload. This could make data
unavilable on the server.

For #7410
2019-09-02 09:58:59 +02:00
Christian Kamm
b3a3f17d74 SyncEngine: Don't duplicate fatal errors
Previously fatal error texts were duplicated: Once they entered the
SyncResult via the SyncFileItem and once via syncError().

syncError is intended for folder-wide sync issues that are not pinned
to particular files. Thus that duplicated path is removed.

For #5088
2019-07-31 09:02:02 +02:00
Christian Kamm
c44ca94e07 Vfs: Add hook to allow update-metadata for unchanged files
Allows winvfs to convert files to placeholders when vfs is enabled. This
is required to mark files as in-sync #7329.
2019-07-24 10:50:39 +02:00
Christian Kamm
6b89747a26 VfsSuffix: Wipe stale pin states #7273
Previously the pin states of deleted files stayed in the 'flags'
database and could be inadvertently reused when a new file with the same
name appeared. Now they are deleted.

To make this work right, the meaning of the 'path' column in the 'flags'
table was changed: Previously it never had the .owncloud file suffix.
Now it's the same as in metadata.path.

This takes the safe parts from #7274 for inclusion in 2.6. The more
elaborate database schema changes (why use 'path' the join the two
tables in the first place?) shall go into master.
2019-06-27 15:57:20 +02:00
Olivier Goffart
819175ad89 SyncEngine: Fix renaming a single file cause the "delete all file" popup
Possibly a regression, since the new discovery discovers rist the renamed
files as removed

Issue #7204
2019-06-04 15:56:35 +02:00
Christian Kamm
03b65e210b Vfs: Mark sqlite temporaries excluded on db-open #7141
The previous patch ensured that the sqlite temporaries weren't deleted
and recreated for every sync run, but there was still time between
client startup and the first sync run where they would have the
"needs-sync" icon.
2019-04-26 13:51:18 +02:00
Christian Kamm
2a900901d3 SyncEngine: Don't close db when done #7141
The db-close operation is likely a leftover from when the SyncEngine
owned its own db connection and serves no purpose anymore.

Closing the db causes the removal of the temporary wal and shm files.
These files are recreated when the db is opened again, which happens
almost immediately.

This is a problem for winvfs because the delete-recreate step wipes the
exclusion state on these files just after the sync is done. That meant
that the db temporaries permanently had a "needs sync" icon marker shown
in the explorer.

Avoiding reopening the db also reduces the number of log messages per
sync.
2019-04-16 13:41:24 +02:00
Christian Kamm
702a494cbb PropagateIgnore: Default to NormalError for INSTRUCTION_ERROR
Previously if one set the instruction to ERROR while forgetting to set
an error status, it'd propagate as FileIgnored. Now the default is
NormalError for INSTRUCTION_ERROR and FileIgnored for
INSTRUCTION_IGNORE.
2019-04-11 13:48:56 +02:00
Christian Kamm
a627f373b1 Ensure local discovery on selective sync changes
As far as I'm aware local discovery can be skipped on folders that are
selective-sync blacklisted, so a local discovery is required when an
entry is removed from the blacklist.

Also rename
avoidReadFromDbOnNextSync() -> schedulePathForRemoteDiscovery()
since the old name might also imply it's not read from db in the local
discovery - which is not the case. Use Folder::
schedulePathForLocalDiscovery() for that.
2019-02-14 10:57:09 +01:00
Christian Kamm
89753773e3 Fix warnings about signedness
Sizes are always qint64, not unsigned.
TransferIds are always uint.
2019-02-14 10:53:33 +01:00
Christian Kamm
010abe4c82 Pin state updates
- unspecified and inherited are different
- move enum to header in common/
- access through Vfs instead of directly in Journal
2019-02-11 13:35:14 +01:00
Christian Kamm
5680950f8d Vfs: Clarify SyncEngine::wipeVirtualFiles()
Possibly the behavior should actually change and the function should
de-placeholder all items, not just dehydrated ones.
2019-02-11 13:35:14 +01:00
Christian Kamm
e0200fbbd2 Discovery: Add signal for silentlyExcluded files
This allows SyncFileStatusTracker to also know about these. After all
its information is used to provide icons for them too.
2019-02-11 13:35:14 +01:00
Christian Kamm
06b4437527 Vfs: Make files that end up in db placeholders
Since 'placeholder' just means that it's an item of the special type
that the vfs plugin can deal with - no matter whether hydrated or
dehydrated - all done items should become placeholders. Even
directories.

Now every file that passes through updateMetadata() will be converted to
a placeholder if necessary.
2019-02-11 13:35:14 +01:00
Christian Kamm
e03df74965 Move SyncFileStatus to libcommon
It'll be needed in vfs plugins so they can connect to the data coming
out of SyncFileStatusTracker.
2019-02-11 13:35:14 +01:00
Christian Kamm
0ed8d32ddb File watcher: Reduce touch ignore duration
On Linux and Windows the file watcher can't distinguish between changes
that were caused by the process itself, like during a sync operation,
and external changes. To work around that the client keeps a list of
files it has touched and blocks notifications on these files for a bit.

The duration of this block was originally and arbitrarily set at 15
seconds. During manual tests I regularly thought there was a bug when
syncs didn't trigger, when the only problem was that my changes happened
too close to a previous sync operation.

This change reduces the duration to three seconds. I imagine that this
is still enough.

Also use std::chrono while at it.
2019-02-10 10:41:01 +01:00
Christian Kamm
c5705de4cc SyncEngine: Rename phase finishing slots
slotDiscoveryJobFinished -> slotDiscoveryFinished
slotFinished -> slotPropagationFinished

This should be clearer. Particular the
slotFinished -> finalize -> emit finished()
chain was confusing before.
2019-01-18 11:16:14 +01:00
Christian Kamm
28dcceb3e4 Discovery: Fix aborting during discovery #6972 2019-01-18 11:16:14 +01:00
Christian Kamm
55eda59d05 Discovery: Ensure selective sync lists are sorted #6958 2019-01-18 11:00:13 +01:00
Olivier Goffart
3313de9066 Discovery: Handle the blacklistFiles from the server capabilities
Issue #434

Ideally one could add the blacklist to the exlucde reggexp, but this
is simpler
2018-12-22 18:19:37 +01:00
Olivier Goffart
3b0a21028c SyncEngine: Ensure that the paths passed to the discovery ends with slashes
This was making the tx.pl test fail
2018-12-18 11:06:31 +01:00
Olivier Goffart
9fbef38247 Merge remote-tracking branch 'owncloud/vfs-master'
Conflicts:
	src/libsync/filesystem.cpp
	src/libsync/filesystem.h
2018-12-07 10:04:43 +01:00
Olivier Goffart
c3277331eb Merge branch 'readonly-restore-context-6252' into 'master'
Conflicts:
	src/gui/socketapi.cpp
	src/gui/socketapi.h
	src/libsync/syncengine.cpp
2018-12-03 14:18:35 +01:00
Christian Kamm
4f56366fd5 vfs: Ensure SyncOptions::_vfs is never null
- Create a VfsOff derived class
- Make it a shared pointer shared with Folder::_vfs
2018-11-26 13:02:48 +01:00
Christian Kamm
29a52e9600 vfs: Reenable local metadata updating of vfs files 2018-11-26 13:02:48 +01:00
Christian Kamm
06928a4037 winvfs: initial work
Done by ckamm and dschmidt
2018-11-26 12:53:30 +01:00
Olivier Goffart
682cc6f199 Sync: optimize by removing setFileRecordMetadata
Inh most case we already have a record from before, so avoid doing a useless
lookup in the database.
In owncloudpropagator.cpp, directories do not have a checksum so no need
to call a function that preserves it
2018-11-15 07:26:44 +01:00
Olivier Goffart
69251ed83e Discovery: fix double emission of finished in case of error 2018-11-08 11:22:03 +01:00
Christian Kamm
2a5fbd1913 Discovery: Don't rebuild invalidFilname regex each call 2018-10-19 10:51:25 +02:00
Olivier Goffart
2e16f654ea Merge remote-tracking branch 'owncloud/master' into new_discovery_algo
Conflicts:
	src/csync/csync_update.cpp
        test/csync/csync_tests/check_csync_update.cpp
2018-10-18 13:22:29 +02:00
Olivier Goffart
7daf393c1c Merge remote-tracking branch 'origin/2.5'
Conflicts:
	ChangeLog
2018-10-18 13:05:34 +02:00
Olivier Goffart
20e6ac05e4 Remove the backup deteciton code which was used for server < 9.1 2018-10-16 16:08:49 +02:00
Olivier Goffart
2df0fd936a New discovery algorithm: fixups
Adapt reviews from ckamm in https://github.com/owncloud/client/pull/6738#pullrequestreview-164623532

- SyncJournalFileRecord: initialize everything inline
- Add more comments
- And some ENFORCE
2018-10-15 18:59:09 +02:00
Olivier Goffart
6cb57cf439 Local discovery: always recurse within touched directory
If the file system watcher tells us a directory was modified, we should
recurse into it because it means it is probably a new directory

Issue #6804
2018-10-12 12:10:50 +02:00
Olivier Goffart
d62027db4a New Discovery algorithm: Remove the sync cleanup phase
Since we do not recurse within some directories, many files are not seen.

The stale entry will cleanup by themself as the sync engine try to remove
the files that are already removed.
Should we need to actually do this cleanup, it should be dotected in the
discovery.
2018-10-09 15:54:42 +02:00
Olivier Goffart
442a0e486f New Propagation algorithm: Fetch, and emit, the root etag
Remove the feature to concatenate etags as servers that don't
have a root etag are no longer suported
2018-10-09 15:12:02 +02:00
Olivier Goffart
28f1c95b96 Merge remote-tracking branch 'origin/master' into new_discovery_algo
Conflicts:
	src/csync/csync_reconcile.cpp
	src/csync/csync_update.cpp
	src/libsync/syncengine.cpp
	src/libsync/syncengine.h

Note: csync changes from 5e442f588e
are not included and will be fixed separately
2018-10-04 11:23:19 +02:00
Olivier Goffart
0290564d5c Merge remote-tracking branch 'owncloud/master' into delta-sync
Conflicts:
	.gitmodules
	src/cmd/cmd.cpp
	src/gui/generalsettings.ui
	src/libsync/propagatedownload.cpp
	src/libsync/propagateuploadng.cpp
2018-09-14 15:25:59 +02:00
Christian Kamm
b4f63ca219 Conflicts: Rename BasePath to InitialBasePath #6709 2018-08-22 09:03:54 +02:00
Olivier Goffart
67ee64a769 New discovery algo: data finger print 2018-08-21 19:54:48 +02:00
Olivier Goffart
597c28f316 Merge remote-tracking branch 'master' into new_discovery_algo 2018-08-21 11:22:38 +02:00
Olivier Goffart
c85cd8e225 Conflict handling: add the OC-ConflictBasePath header
Issue #6702
2018-08-19 21:22:08 +02:00
Olivier Goffart
4a3fce531c SyncEngine: Fix renames making hierarchy inversion
Issue #6694
2018-08-16 10:38:18 +02:00
Olivier Goffart
58f46010b5 SyncEngine: Fix the "direction" of the "all file delted" message when the server is reset
Using the direction of the "first" item is not enough as it might be a directory
which is a UPDATE_META_DATA,  or there can have been a few changes locally as well.

As reported on https://github.com/owncloud/client/issues/6317#issuecomment-412163113
2018-08-15 09:34:19 +02:00
Olivier Goffart
58d6632eaf Data-Fingerprint: Fix backup detection when fingerprint is empty
Add a test to test the data fingerprint feature make me realize it was broken.
The code was relying in the distinction between empty and null QByteArray,
but this was a bad idea as this difference is lost when going through QString.
2018-08-14 10:57:20 +02:00
Olivier Goffart
b6db624198 New discovery algorithm: Call FolderDiscovered 2018-08-06 12:53:51 +02:00
Olivier Goffart
2e82518516 New Discovery algorithm: Refactor a bit the way the signal are emited 2018-08-06 12:41:59 +02:00
Olivier Goffart
926c27a73d New Discovery Phase: fix a few FIXME
Some FIXME were already fixed.
2018-08-06 11:30:21 +02:00
Olivier Goffart
eb669c997e More cleanup of csync remains 2018-07-26 13:55:02 +02:00