OAuth2 access token typically only has a token valid for 1 hour.
Before this patch, when the token was timing out during the sync, the
sync was aborted, and the ConnectionValidator was then requesting a new
token, so the sync can be started over.
If the discovery takes longer than the oauth2 validity, this means that
the sync can never proceed, as it would be always restarted from scratch.
With this patch, we try to transparently renew the OAuth2 token and restart
the jobs that failed because the access token was invalid.
Note that some changes were required in the GETFile job because it handled
the error itself and so it was erroring the jobs before its too late.
Issue #6814
If the server does not set the mtime, it is not a big problem for the
synchronisation.
The test was used before so we could do a PROPPATCH for server that did not
support this header. But now that all server supports that we don't need to
to the check. (We do not do the PROPPATCH since we got rid of the neon
dependency)
Apparently, it may happen that some backend don't support setting mtime
and this can lead to this error.
https://github.com/owncloud/client/issues/6797
We need to use the user id to check if we are connected to the right account.
These might be different from the HTTP Basic Auth login. (LDAP setups)
When the account was configured as an oauth2 account form the wisard, the
http_user was already set correctly to the user id. But when the server is
upgrading from basic auth to oauth2, we need to pick the right login.
Note that Account::davUser() already defaults to the HTTP user when none
is set, so this means the upgrade will be fine if this is not set in the
config.
Issues:
https://github.com/owncloud/oauth2/issues/109https://github.com/owncloud/enterprise/issues/2781
Unfortunately to do this, the local update phase must write to the
database, creating a new side-effect and order dependency (local update
must run before remote update).
Fixes two bugs that appeared since the introduction of the struct:
- when reading permissions from the journal, null ("") was read as
empty-not-null
- when reading permissinos from the server, empty ("") was read as null
Addresses #4608
Add a test to test the data fingerprint feature make me realize it was broken.
The code was relying in the distinction between empty and null QByteArray,
but this was a bad idea as this difference is lost when going through QString.
The server reply with a code 400 when the token is invalid,
the client was understanding this error as a network error, and was retying
again with the same token.
Instead, we must rely on what the json is saying, even if the reply is
not a 200 code.
Issue https://github.com/owncloud/enterprise/issues/2777
The check was added for #6317 in commit
13eb64584f.
We did see missing mtimes in replies in tests with live servers though.
Possibly those were old incomplete responses cached in the stat cache?
This is about the conflicts that happens when the file has been uploaded
correctly to the server, but the etag was not recieved because the connection
was closed before we got the reply.
We used to compare only the mtime when comparing the uploaded file and the
existing file. However, to be perfectly correct, we also should check the
size.
This was found because TestChunkingNG::connectionDroppedBeforeEtagRecieved is
flaky. Example of faillure found in https://drone.owncloud.com/owncloud/client/481/5
while testing PR #6626
(very trimmed log:)
06-29 07:58:02:015 [ info sync.csync.csync ]: ## Starting local discovery ##
06-29 07:58:02:016 [ info sync.csync.updater ]: Database entry found, compare: 1530259082 <-> 1530259051, etag: <-> 1644a8c8750, inode: 1935629 <-> 1935629, size: 301 <-> 300, perms: 0 <-> ff, type: 0 <-> 0, checksum: <-> SHA1:cc9adedebe27a6259efb8d6ed09f4f2eff559ad1, ignore: 0
06-29 07:58:02:016 [ info sync.csync.updater ]: file: A/a0, instruction: INSTRUCTION_EVAL <<=
06-29 07:58:02:972 [ warning sync.networkjob ]: QNetworkReply::NetworkError(OperationCanceledError) "Connection timed out" QVariant(Invalid)
.. next sync...
06-29 07:58:02:980 [ info sync.engine ]: #### Discovery start ####################################################
06-29 07:58:02:981 [ info sync.csync.csync ]: ## Starting local discovery ##
06-29 07:58:02:983 [ info sync.csync.updater ]: Database entry found, compare: 1530259082 <-> 1530259051, etag: <-> 1644a8c8750, inode: 1935629 <-> 1935629, size: 302 <-> 300, perms: 0 <-> ff, type: 0 <-> 0, checksum: <-> SHA1:cc9adedebe27a6259efb8d6ed09f4f2eff559ad1, ignore: 0
06-29 07:58:02:983 [ info sync.csync.updater ]: file: A/a0, instruction: INSTRUCTION_EVAL <<=
06-29 07:58:02:985 [ info sync.csync.csync ]: ## Starting remote discovery ##
06-29 07:58:02:985 [ info sync.networkjob ]: OCC::LsColJob created for "http://localhost/owncloud" + "" "OCC::DiscoverySingleDirectoryJob"
06-29 07:58:02:987 [ info sync.csync.updater ]: Database entry found, compare: 1530259082 <-> 1530259051, etag: 1644a8c8b26 <-> 1644a8c8750, inode: 0 <-> 1935629, size: 301 <-> 300, perms: ff <-> ff, type: 0 <-> 0, checksum: SHA1:5adcdac9608ae0811247f07f4cf1ab0a2ef99154 <-> SHA1:cc9adedebe27a6259efb8d6ed09f4f2eff559ad1, ignore: 0
06-29 07:58:02:987 [ info sync.csync.updater ]: file: A/a0, instruction: INSTRUCTION_EVAL <<=
06-29 07:58:02:989 [ info sync.csync.csync ]: Update detection for remote replica took 0.004 seconds walking 13 files
06-29 07:58:02:990 [ info sync.engine ]: #### Discovery end #################################################### 9 ms
06-29 07:58:02:990 [ info sync.database ]: Updating file record for path: "A/a0" inode: 1935629 modtime: 1530259082 type: 0 etag: "1644a8c8b26" fileId: "16383ea4" remotePerm: "WDNVCKR" fileSize: 301 checksum: "SHA1:cc9adedebe27a6259efb8d6ed09f4f2eff559ad1"
06-29 07:58:02:990 [ info sync.csync.reconciler ]: INSTRUCTION_UPDATE_METADATA client file: A/a0
06-29 07:58:02:990 [ info sync.csync.csync ]: Reconciliation for local replica took 0 seconds visiting 13 files.
06-29 07:58:02:990 [ info sync.csync.reconciler ]: INSTRUCTION_UPDATE_METADATA server dir: A
06-29 07:58:02:990 [ info sync.csync.csync ]: Reconciliation for remote replica took 0 seconds visiting 13 files.
06-29 07:58:02:990 [ info sync.engine ]: #### Reconcile end #################################################### 9 ms
06-29 07:58:02:990 [ info sync.database ]: Updating local metadata for: "A/a0" 1530259082 302 1935629
FAIL! : TestChunkingNG::connectionDroppedBeforeEtagRecieved(small file) '!fakeFolder.syncOnce()' returned FALSE. ()
Installing to lib/${APPLICATION_EXECUTABLE} has caused a bunch of
irritations in the past and subtle annoying to fix bugs. To avoid name
clashes with branded clients ${APPLICATION_EXECUTABLE} becomes now
part of the filename instead of the subfolder.
The concrete motivation to change this now is that on Windows there
is no RPATH and it's not possible to run owncloud directly from the
Craft Root folder, which is nice when you're developing on Windows.
It would have been possible to change this just for Windows but as
written earlier this has caused lots of issues and thus I think it's
a good idea to just stay consistent accross platforms when touching it.
It could happen that readyRead was emitted for incoming data while the
download was not yet finished. Then the network job could finish with
no more data arriving - so readyRead wasn't emitted again.
To fix this, the finished signal also gets connected to the readyRead
slot.
The problem here is that we were sometimes allocating the error_string with
qstrdup, which need to be released with delete[] and not free().
Simplify the code by using QString instead.
```
==7230== Mismatched free() / delete / delete []
==7230== at 0x4C2E10B: free (vg_replace_malloc.c:530)
==7230== by 0x57C2321: csync_s::reinitialize() (csync.cpp:247)
==7230== by 0x548130F: OCC::SyncEngine::finalize(bool) (syncengine.cpp:1212)
==7230== by 0x5481223: OCC::SyncEngine::handleSyncError(csync_s*, char const*) (syncengine.cpp:746)
==7230== by 0x5483E78: OCC::SyncEngine::slotDiscoveryJobFinished(int) (syncengine.cpp:965)
==7230== by 0x5495E34: QtPrivate::FunctorCall<QtPrivate::IndexesList<0>, QtPrivate::List<int>, void, void (OCC::SyncEngine::*)(int)>::call(void (OCC::SyncEngine::*)(int), OCC::SyncEngine*, void**) (qobjectdefs_impl.h:134)
==7230== by 0x5495D92: void QtPrivate::FunctionPointer<void (OCC::SyncEngine::*)(int)>::call<QtPrivate::List<int>, void>(void (OCC::SyncEngine::*)(int), OCC::SyncEngine*, void**) (qobjectdefs_impl.h:167)
==7230== by 0x5495CB5: QtPrivate::QSlotObject<void (OCC::SyncEngine::*)(int), QtPrivate::List<int>, void>::impl(int, QtPrivate::QSlotObjectBase*, QObject*, void**, bool*) (qobjectdefs_impl.h:396)
==7230== by 0xA9BF2E1: QObject::event(QEvent*) (in /usr/lib/libQt5Core.so.5.11.0)
==7230== by 0x64BE983: QApplicationPrivate::notify_helper(QObject*, QEvent*) (in /usr/lib/libQt5Widgets.so.5.11.0)
==7230== by 0x64C625A: QApplication::notify(QObject*, QEvent*) (in /usr/lib/libQt5Widgets.so.5.11.0)
==7230== by 0xA994BC8: QCoreApplication::notifyInternal2(QObject*, QEvent*) (in /usr/lib/libQt5Core.so.5.11.0)
==7230== Address 0x225b2640 is 0 bytes inside a block of size 50 alloc'd
==7230== at 0x4C2DC6F: operator new[](unsigned long) (vg_replace_malloc.c:423)
==7230== by 0xA7E8FC8: qstrdup(char const*) (in /usr/lib/libQt5Core.so.5.11.0)
==7230== by 0x53F5750: OCC::DiscoveryJob::remote_vio_opendir_hook(char const*, void*) (discoveryphase.cpp:666)
==7230== by 0x57E1278: csync_vio_opendir(csync_s*, char const*) (csync_vio.cpp:39)
==7230== by 0x57D718F: csync_ftw(csync_s*, char const*, int (*)(csync_s*, std::unique_ptr<csync_file_stat_s, std::default_delete<csync_file_stat_s> >), unsigned int) (csync_update.cpp:674)
==7230== by 0x57C1B05: csync_update(csync_s*) (csync.cpp:109)
==7230== by 0x53F5BCC: OCC::DiscoveryJob::start() (discoveryphase.cpp:718)
==7230== by 0x54B8F74: OCC::DiscoveryJob::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) (moc_discoveryphase.cpp:494)
==7230== by 0xA9BF2E1: QObject::event(QEvent*) (in /usr/lib/libQt5Core.so.5.11.0)
==7230== by 0x64BE983: QApplicationPrivate::notify_helper(QObject*, QEvent*) (in /usr/lib/libQt5Widgets.so.5.11.0)
==7230== by 0x64C625A: QApplication::notify(QObject*, QEvent*) (in /usr/lib/libQt5Widgets.so.5.11.0)
==7230== by 0xA994BC8: QCoreApplication::notifyInternal2(QObject*, QEvent*) (in /usr/lib/libQt5Core.so.5.11.0)
==7230==
```
We want to keep the body so we can get the message from it
(Issue #6459)
TestDownload::testErrorMessage did not fail because the FakeErrorReply
did not emit readyRead and did not implement bytesAvailable.
Otherwise a interrupted or unsuccessful download would mean that the
download-intend was forgotten. The next sync would reestablish the
virtual file instead.
Also, if there is too-new configuration, backup the file, show a
warning message asking the user whether it's ok to discard the
configuration from the future.
See #6504
When owncloud is restored, at boot time, it might be started before the
crendential manager. So if we detect an error, wait 10 seconds and hopefully
it'd be loaded by then.
Issues: #4274, #6522
Some servers have virus scanners and the like that can delay the
response of the final chunked upload assembly significantly, often
breaking the current 5min (!) timeout. See owncloud/enterprise#2480
for details.
