will prevent nextcloudcmd command line client from ignoring the settings
handled by SyncOptions
Signed-off-by: Matthieu Gallien <matthieu.gallien@nextcloud.com>
the new method added to query the db is not working and so the fix for
vfs is executed at each sync run
the new method for bool was not really needed so let's just remove it
(and that will make the usage of SqlQuery be correct
Signed-off-by: Matthieu Gallien <matthieu.gallien@nextcloud.com>
In the past not all files were converted to placeholder files when
converting an existing sync folder to a virtual files folder. Because
some files were not converted to placeholder files, the status would
be wrong on the files. This code makes sure that every file in a
virtual files folder is a placeholder file. It could be removed in the
future.
Signed-off-by: Felix Weilbach <felix.weilbach@nextcloud.com>
While investigating #3172, I discoverd that the sync will hang forever
if the user has deleted all files on the server. Calling the callback
resolves the problem.
Signed-off-by: Felix Weilbach <felix.weilbach@nextcloud.com>
For a usual file sync event we check for actual changes in the local file,
after an unlock the local file might be unchanged so we need to sync it anyhow.
Fixes: owncloud/enterprise#3609
Previously the pin states of deleted files stayed in the 'flags'
database and could be inadvertently reused when a new file with the same
name appeared. Now they are deleted.
To make this work right, the meaning of the 'path' column in the 'flags'
table was changed: Previously it never had the .owncloud file suffix.
Now it's the same as in metadata.path.
This takes the safe parts from #7274 for inclusion in 2.6. The more
elaborate database schema changes (why use 'path' the join the two
tables in the first place?) shall go into master.
The previous patch ensured that the sqlite temporaries weren't deleted
and recreated for every sync run, but there was still time between
client startup and the first sync run where they would have the
"needs-sync" icon.
The pin state is a per-item attribute that has an effect on _type:
AlwaysLocal dehydrated files will be marked for hydration and OnlineOnly
hydrated files will be marked for dehydration.
Where exactly this effect materializes depends on how the pin states are
stored. If they're stored in the db (suffix) the dbEntry._type is
changed during the discovery.
If the pin state is stored in the filesystem, the localEntry._type must
be adjusted by the plugin's stat callback.
This patch makes pin states behave more consistently between plugins.
Previously with suffix-vfs pin states only had an effect on new remote
files. Now the effect of pinning or unpinning files or directories is as
documented and similar to other plugins.
As far as I'm aware local discovery can be skipped on folders that are
selective-sync blacklisted, so a local discovery is required when an
entry is removed from the blacklist.
Also rename
avoidReadFromDbOnNextSync() -> schedulePathForRemoteDiscovery()
since the old name might also imply it's not read from db in the local
discovery - which is not the case. Use Folder::
schedulePathForLocalDiscovery() for that.
- SyncJournalDB functions now behind internalPinStates() to avoid
accidental usage, when nearly everyone should go through Vfs.
- Rename Vfs::getPinState() to Vfs::pinState()
supportsSelectiveSync(): clearer than !supportsVirtualFiles() and allows
extra logic
isVfsOnOffSwitchPending(): Somewhat awkward way of dealing with the
phase between a user requesting vfs state to be switched and it
actually happening
This allows enabling and disabling vfs.
To distinguish this operation from setting the root pin state, the
availability setting is adjusted as well to be similar to the
menu that shows in the shell extensions.
This is to avoid issues on OSX, where the ._ prefix has special meaning.
Originally (before 2.3.2) ._ was necessary to guarantee exclusion. But
since then the .sync_ prefix is excluded as well.
This does not affect existing database files.
Previously there'd likely be a mess if a 2.6 winvfs folder was attempted
to be used with a 2.5 client. Now the older clients will ignore these
folders.
This helps support 2.5 settings where there are virtual files in the
tree but new files aren't created virtual.
It's also a prelude for #6815
There's currently no way of
- upgrading vfs plugins (a silent suffix->winvfs upgrade is attempted
once only, when moving to master)
- disabling vfs capabilities outright
Also, if there is too-new configuration, backup the file, show a
warning message asking the user whether it's ok to discard the
configuration from the future.
See #6504
Now the db entries for placeholders will have the full placeholder
paths. That way older clients will, on remote discovery, delete the
placeholders and download the real files.
Previously it depended on addFolder() / removeFolder() calls to adjust
watchers when new folders were added or removed. There also needed to be
complex move handling.
Now, any folder creation/move-in notifications automatically trigger
watcher additions and folder deletion/move-out triggers removal.
We're not interested in any found element in these cases, just to check
that none of the elements matches.
Signed-off-by: Harald Eilertsen <haraldei@anduin.net>
It filters the error out of the list of blocking errors. It now shows up
in the Activities and Notificattions list as a warning.
Signed-off-by: Camila San <hello@camila.codes>
We otherwise normalize all path in the C form, so we must have
the Folder's path normalized the same. Or all comparizon will fail
(such as knowing if a file from the SocketAPI or the FilesystemWatcher
are part of the folder)
Issue #4424
If the SyncResult incorrectly believes that there are no conflicts, the
tray icon won't be correct and there will be no warning about unresolved
conflicts on the account.
Nevertheless, it's pretty awkward that the IssuesWidget is better
informed about pending conflicts than the Folder itself. This kind of
backwards data flow is very confusing.
Unfortunately the only alternative I see is to either keep track of
this information in two places (also in Folder), or create a common
data-holding class that can serve as a model instance for the issues
view as well as provide data directly to the Folder - which would
have been a much larger change.
- Changes the configuration name in ConfigFile and GeneralSettings
accordingly with the new text.
- Makes sure the user sees error and conflict messages even if the
setting is disabled.
Signed-off-by: Camila San <hello@camila.codes>
Make ExcludedFiles something that is instantiated outside of
the CSYNC context and then given to it as a hook.
ExcludedFiles still lives in csync_exclude and the internal
workings haven't been touched.
... even if the file is not changed.
We get an UPDATE_METADATA in that case, so make sure we let the
SyncFileStatusTracker know about it.
That means we need to filter out UPDATE_METADATA in the other listeners
of this signal.
Issue #6098
We mostly trust the file watchers meaning that we don't re-scan the
local tree if we have done that recently and no file watcher events
have arrived. If the file watchers invalidate a subtree, we rescan
only that subtree.
Since we're not entirely sure the file watchers are reliable, we still
do full local discoveries regularly (1h by default). There is a config
file setting as well as an environment variable to control the interval.
This makes it unnecessary for FolderMan to manage the list and removes
the need for some forwarders.
This is done in preparation for follow-up commits that want to add
diagnostics to FolderWatcher that shall be available from within Folder.
This gets rid of the csync_statedb sqlite layer and use
the same code and same connection as the rest of the SyncEngine.
Missing functions are added to SyncJournalDb and change a few minor
things (like changing SyncJournalFileRecord::_modtime to be an int64
instead of a QDateTime, like it was in csync).
The current implementation would return the same value whether the query failed
or if no row would be found. This is something that is currently checked by csync
and needs to be provided if we want to use SyncJournalDB there.
Adjusted all call sites to also check the return value even though they
could still just rely on rec.isValid(), but makes it more explicit as to what
happens for database errors in those cases, if we ever want to gracefully handle
them.
This is motivated by the fact that QMetaObject::noralizeSignature takes 7.35%
CPU of the LargeSyncBench. (Mostly from ABstractNetworkJob::setupConnections and
PropagateUploadFileV1::startNextChunk). It could be fixed by using normalized
signature in the connection statement, but i tought it was a good oportunity
to modernize the code.
This commit only contains calls that were automatically converted with clazy.
Now that csync builds as C++, this will avoid having to implement
functionalities needed by csync mandatorily in csync itself.
This library is built as part of libocsync and symbols exported
through it.
This requires a relicense of Utility as LGPL. All classes moved into
this library from src/libsync will need to be relicensed as well.
We only want to know if they were touched within the last 15 seconds,
so change the data structure to use a QMultiMap, and sort them by
QElapsedTimer. This allows us to iterate over old entries ordered by
time and to stop once we find a recent entry.
This makes the look-up slower but in most cases the folder watcher
will report any change within milliseconds, and we start from the
most recent. What this really makes slower are actual user file
changes while a fast sync is underways which will need to iterate
over the whole map to find out the file isn't there.
This reduces the growth of the memory usage when downloading a large
amount of files.
Since these errors are blacklisted, it can take up to 24h to retry items
that had a 507 error for a while. This way users can intervene and cause
an upload attempt immediately.
* A bunch of code was determining sync status by ad-hoc comparing some
progress info fields. It can now just check the status, making it
easier to comprehend.
* There's a clear indication for "a new sync is starting", which helps
wiping the issues tab at the right time.
For now we use them for:
* csync errors: This allows them to appear in the sync issues tab
* insufficient local disk space, as a summary of individual file errors
Insufficient remote space will use them too, as might other issues that
are bigger than a single sync item.
When synchronizing a folder on a samba share, creating files that begin
with ._ is often forbidden. This prevented the client from creating
its ._sync_abcdef.db file.
Now, it'll check whether the preferred filename is creatable, and if
it isn't it'll use .sync_abcdef.db instead.
The disadvantage is that this alternative path won't be ignored by
older clients - that was the reason for the ._ prefix.
Use qCInfo for anything that has general value for support and
development. Use qCWarning for any recoverable error and qCCritical
for anything that could result in data loss or would identify a serious
issue with the code.
Issue #5647
This gives more insight about the logs and allow setting fine-tuned
logging rules. The categories are set to only output Info by default
so this allows us to provide more concise logging while keeping the
ability to extract more information for a specific category when
developping or debugging customer issues.
Issue #5647
* make target duration a client option instead of a capability
* simplify algorithm for determining chunk size significantly
* preserve chunk size for the whole propagation, not just per upload
* move options to SyncOptions to avoid depending on ConfigFile
in the propagator
* move chunk-size adjustment to after a chunk finishes, not when
a new chunk starts
When there is no errors _syncResult.firstItemError() is NULL, and accessing
it's _file member is an undefined behavior. (Thankfully, createGuiLog did not
use the string when the count was 0, but we are not supposed to create
null references.
Found with the UB sanitizer:
src/gui/folder.cpp:348:49: runtime error: member access within null pointer of type 'OCC::SyncFileItem'
src/gui/folder.cpp:348:19: runtime error: reference binding to null pointer of type 'const QString'
