I am running into file lock error
[TileDB::IO] Error: Cannot open filelock '/tmp/Rtmpymp9pr/file751663dcef39/CytoTrol_CytoTrol_1.fcs.tile/params/__lock.tdb'; Permission denied
The reason is the tiledb directory I am trying to read is created by another person and I only have readonly permission on that by default.
I am currently on linux posix.
My question is if tiledb always requires write permission on the file in order to read its content.
And I am aware of https://github.com/TileDB-Inc/TileDB/issues/1260. But wonder if it is generally recommended to disable the lock even when I just want to be able to do the tiledb read operations.
Thanks for the bug report. The following command will print the unix permissions for the __lock.tdb file and each directory leading to it. Please run this in your linux shell and copy the output here. If the __lock.tdb no longer exists, run the command on the path without the filename. This will help me debug the issue and reproduce it on my end. Thanks!
namei -l /tmp/Rtmpymp9pr/file751663dcef39/CytoTrol_CytoTrol_1.fcs.tile/params/__lock.tdb
@mikejiang I’ve submitted a speculative fix for review: https://github.com/TileDB-Inc/TileDB/pull/1676
The fix should make it into the next minor release, but you can cherry-pick that commit if you’re building from source.
Thanks for the quick fix. I will give it a try.
Here is the output that you asked for
namei -l /tmp/Rtmpymp9pr/file751663dcef39/CytoTrol_CytoTrol_1.fcs.tile/params/__lock.tdb
f: /tmp/Rtmpymp9pr/file751663dcef39/CytoTrol_CytoTrol_1.fcs.tile/params/__lock.tdb
drwxr-xr-x root root /
drwxrwxrwt root root tmp
drwx------ wjiang2 wjiang2 Rtmpymp9pr
dr-xr-xr-x wjiang2 wjiang2 file751663dcef39
dr-xr-xr-x wjiang2 wjiang2 CytoTrol_CytoTrol_1.fcs.tile
dr-xr-xr-x wjiang2 wjiang2 params
-r-xr-xr-x wjiang2 wjiang2 __lock.tdb
Hi Mike,
Joe had you do show the pertinent details. The problem here is the access to the top-level temporary directory created by R, your /tmp/Rtmpymp9pr which has mode 0700 meaning only user wjiang2 can change into it and read it. As you are not wijiang2 you get an error.
There are two simply ways to mitigate. First, given that the other permissions within the directory were already changed, have wijiang2 also change the remaining top-level directory. Second, the TileDB array could of course be written in /tmp/ rather than in a temporary R-session directory. So if there is control over how CytoTrol_CytoTrol_1.fcs.tile is written, maybe the path used in the array creation could be changed?
Let us know if we can help some more.
Best regards, Dirk
Sorry about the misleading info I provided in my initial post. I am actually wjiang2, so do have read permission. The error was caught by my coworker initially when I shared one tiledb folder to him where he only has read permission.
So to reproduce his issue, I went ahead write a test case in R and intentionally strip my own write permission by chmod -R u-w /tmp/Rtmpymp9pr/file751663dcef39.
But thanks a lot for the response!
The issue remains the same:
drwx------ wjiang2 wjiang2 Rtmpymp9pr
No group, no world, meaning nobody apart from wjiang2 can access this. That is a feature of the R temporary directory. I still suspect that you inadvertently started in the wrong corner by creating a resource you plan to share inside of a by design private directory. My preference would be to create the to-be-shared data somewhere else.
Hope this helps, Dirk
Thanks for sharing Mike – the patch I mentioned earlier should fix your problem. If you do try the patch and still see the issue, please let me know.
Joe,
The problem remains after I pull the latest dev branch.
commit b85667a79175556a573cb6c20be44f54728e54c8 (HEAD -> dev, origin/dev, origin/HEAD)
Author: Seth Shelnutt <Shelnutt2@gmail.com>
Date: Thu Jun 11 16:10:58 2020 -0400
...
commit 962def8088d46cdd84961791636b874725014eb2
Author: Joe maley <joe@tiledb.io>
Date: Wed Jun 10 11:49:25 2020 -0400
Posix VFS lock file permissions
Here is the line that triggers the error
tiledb::Array array(*ctxptr_, (fs::path(uri_) / "params").string(), TILEDB_READ);
Not sure if there is TILEDB_READONLY flag that should be used?
For now, I still had to do disable lock in order to read the file.
cfg["vfs.file.enable_filelocks"] = "false";
@mikejiang That’s surprising because I’ve tested it on a read-only array. Either way, I’ve pushed a new branch jpm/debug-vfs-read-lock to the repository. It contains a few log statements that will help me debug the issue. Could you pull+build+run this branch and paste the the output here? It should look something like:
1: [2020-06-15 08:17:36.101] [tiledb] [Process: 16599] [Thread: 16599] [error] [DEBUG filelock_lock] filename: /media/psf/Home/Workspace/TileDB_Ubuntu_1804_Workspace/TileDB3/TileDB/test/tiledb_test/string_dims/__lock.tdb shared: 1
1: [2020-06-15 08:17:36.102] [tiledb] [Process: 16599] [Thread: 16599] [error] [DEBUG filelock_lock] rwx------
1: [2020-06-15 08:17:36.102] [tiledb] [Process: 16599] [Thread: 16599] [error] [DEBUG filelock_lock] fd: 6
Never mind, I didn’t set prefix during my last built so I was still using the staled version from /usr/local.
Sorry about such stupid mistake that’s wasting everyone’s time. I swear it won’t happen again.
Anyway, It is all good now.
No problem, thanks for getting back to us. I just wanted to ensure the fix is working. You found a real bug and verified the fix for us, I really appreciate it! Let me know if you have any other questions.
2 Likes