Since 'placeholder' just means that it's an item of the special type
that the vfs plugin can deal with - no matter whether hydrated or
dehydrated - all done items should become placeholders. Even
directories.
Now every file that passes through updateMetadata() will be converted to
a placeholder if necessary.
On Linux and Windows the file watcher can't distinguish between changes
that were caused by the process itself, like during a sync operation,
and external changes. To work around that the client keeps a list of
files it has touched and blocks notifications on these files for a bit.
The duration of this block was originally and arbitrarily set at 15
seconds. During manual tests I regularly thought there was a bug when
syncs didn't trigger, when the only problem was that my changes happened
too close to a previous sync operation.
This change reduces the duration to three seconds. I imagine that this
is still enough.
Also use std::chrono while at it.
slotDiscoveryJobFinished -> slotDiscoveryFinished
slotFinished -> slotPropagationFinished
This should be clearer. Particular the
slotFinished -> finalize -> emit finished()
chain was confusing before.
Inh most case we already have a record from before, so avoid doing a useless
lookup in the database.
In owncloudpropagator.cpp, directories do not have a checksum so no need
to call a function that preserves it
Since we do not recurse within some directories, many files are not seen.
The stale entry will cleanup by themself as the sync engine try to remove
the files that are already removed.
Should we need to actually do this cleanup, it should be dotected in the
discovery.
The problem here is that we were sometimes allocating the error_string with
qstrdup, which need to be released with delete[] and not free().
Simplify the code by using QString instead.
```
==7230== Mismatched free() / delete / delete []
==7230== at 0x4C2E10B: free (vg_replace_malloc.c:530)
==7230== by 0x57C2321: csync_s::reinitialize() (csync.cpp:247)
==7230== by 0x548130F: OCC::SyncEngine::finalize(bool) (syncengine.cpp:1212)
==7230== by 0x5481223: OCC::SyncEngine::handleSyncError(csync_s*, char const*) (syncengine.cpp:746)
==7230== by 0x5483E78: OCC::SyncEngine::slotDiscoveryJobFinished(int) (syncengine.cpp:965)
==7230== by 0x5495E34: QtPrivate::FunctorCall<QtPrivate::IndexesList<0>, QtPrivate::List<int>, void, void (OCC::SyncEngine::*)(int)>::call(void (OCC::SyncEngine::*)(int), OCC::SyncEngine*, void**) (qobjectdefs_impl.h:134)
==7230== by 0x5495D92: void QtPrivate::FunctionPointer<void (OCC::SyncEngine::*)(int)>::call<QtPrivate::List<int>, void>(void (OCC::SyncEngine::*)(int), OCC::SyncEngine*, void**) (qobjectdefs_impl.h:167)
==7230== by 0x5495CB5: QtPrivate::QSlotObject<void (OCC::SyncEngine::*)(int), QtPrivate::List<int>, void>::impl(int, QtPrivate::QSlotObjectBase*, QObject*, void**, bool*) (qobjectdefs_impl.h:396)
==7230== by 0xA9BF2E1: QObject::event(QEvent*) (in /usr/lib/libQt5Core.so.5.11.0)
==7230== by 0x64BE983: QApplicationPrivate::notify_helper(QObject*, QEvent*) (in /usr/lib/libQt5Widgets.so.5.11.0)
==7230== by 0x64C625A: QApplication::notify(QObject*, QEvent*) (in /usr/lib/libQt5Widgets.so.5.11.0)
==7230== by 0xA994BC8: QCoreApplication::notifyInternal2(QObject*, QEvent*) (in /usr/lib/libQt5Core.so.5.11.0)
==7230== Address 0x225b2640 is 0 bytes inside a block of size 50 alloc'd
==7230== at 0x4C2DC6F: operator new[](unsigned long) (vg_replace_malloc.c:423)
==7230== by 0xA7E8FC8: qstrdup(char const*) (in /usr/lib/libQt5Core.so.5.11.0)
==7230== by 0x53F5750: OCC::DiscoveryJob::remote_vio_opendir_hook(char const*, void*) (discoveryphase.cpp:666)
==7230== by 0x57E1278: csync_vio_opendir(csync_s*, char const*) (csync_vio.cpp:39)
==7230== by 0x57D718F: csync_ftw(csync_s*, char const*, int (*)(csync_s*, std::unique_ptr<csync_file_stat_s, std::default_delete<csync_file_stat_s> >), unsigned int) (csync_update.cpp:674)
==7230== by 0x57C1B05: csync_update(csync_s*) (csync.cpp:109)
==7230== by 0x53F5BCC: OCC::DiscoveryJob::start() (discoveryphase.cpp:718)
==7230== by 0x54B8F74: OCC::DiscoveryJob::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) (moc_discoveryphase.cpp:494)
==7230== by 0xA9BF2E1: QObject::event(QEvent*) (in /usr/lib/libQt5Core.so.5.11.0)
==7230== by 0x64BE983: QApplicationPrivate::notify_helper(QObject*, QEvent*) (in /usr/lib/libQt5Widgets.so.5.11.0)
==7230== by 0x64C625A: QApplication::notify(QObject*, QEvent*) (in /usr/lib/libQt5Widgets.so.5.11.0)
==7230== by 0xA994BC8: QCoreApplication::notifyInternal2(QObject*, QEvent*) (in /usr/lib/libQt5Core.so.5.11.0)
==7230==
```
Now the db entries for placeholders will have the full placeholder
paths. That way older clients will, on remote discovery, delete the
placeholders and download the real files.
- Controled by an option.
- New remote files start out as ItemTypePlaceholder, are created with a
.owncloud extension.
- When their db entry is set to ItemTypePlaceholderDownload the next
sync run will download them.
- Files that aren't in the placeholder state sync as usual.
- See test cases in testsyncplaceholders.
Missing:
- User ui for triggering placeholder file download
- Maybe: Going back from file to placeholder?
For conflicts generally as well as new files in read-only directories
the context menu will now present delete and move options.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
When the ClientSideEncryption object is fed, also serialize the
encryption info of the folders inside the metadata table.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
We used to do it when the propagation starts, let's do it even before
the discovery starts. This way we'll have a chance to exploit the
information during the discovery phase.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
Add a test to test the data fingerprint feature make me realize it was broken.
The code was relying in the distinction between empty and null QByteArray,
but this was a bad idea as this difference is lost when going through QString.
Since commit 4dc49ff3, we store an entry in the upload info table even
for non chunked uploads. However, if this fails we don't want to remove
non-existant stale chunks if the upload fails.
Without this commit, we would send a DELETE command to clean non-existant
chunks in the dav/uploads/ namespace.
Before, we only cleared the _avoidReadFromDbOnNextSyncFilter *after* a
sync which meant that we had to sync twice after selective sync setup.
Now, we clear the filter *before* a sync as well which allows the actual
next sync to write the correct etags to the db again - instead of only
the sync after that one.
Also expand on comments and rename _avoidReadFromDbOnNextSyncFilter to
_etagStorageFilter.
Everything is invalidated later: after discovery, not before. In
addition entries that should only be invalidated when new local
discovery is done have that behavior now.
Previously conflicts with a different type on both ends lead to sync
errors. Now they are handled in the expected way: the local item gets
renamed and the remote item gets propagated downwards.
This also adds a unittest for the TYPE_CHANGE case. That one looks like
parts of it might be unified with CONFLICT cases.
If the server has the 'uploadConflictFiles' capability conflict
files will be uploaded instead of ignored.
Uploaded conflict files have the following headers set during upload
OC-Conflict: 1
OC-ConflictBaseFileId: 172489174instanceid
OC-ConflictBaseMtime: 1235789213
OC-ConflictBaseEtag: myetag
when the data is available. Downloads accept the same headers in return
when downloading a conflict file.
In the absence of server support clients will identify conflict files
through the file name pattern and attempt to deduce the base fileid.
Base etag and mtime can't be deduced though.
The upload job for a new conflict file will be triggered directly from
the job that created the conflict file now. No second sync run is
necessary anymore.
This commit does not yet introduce a 'username' like identifier that
automatically gets added to conflict file filenames (to name the files
foo_conflict-Fred-1345.txt instead of just foo_conflict-1345.txt).
Previously, there was csync_ftw_type_e and SyncFileItem::Type. Having
two enums lead to a bug where Type::Unknown == Type::File that went
unnoticed for a good while.
This patch keeps only a single enum.
Make ExcludedFiles something that is instantiated outside of
the CSYNC context and then given to it as a hook.
ExcludedFiles still lives in csync_exclude and the internal
workings haven't been touched.
