Hello!
Is a JavaScript API for TileDB available or planned? I would like to parse TileDB data structures in the browser wither via Ajax or local HTML5 reader.
Thank you so much!
Best,
Daniel
Hello Daniel,
We don’t have a JavaScript API for TileDB yet, but we have received a number of requests for it. We will try to plan its development in the next few months.
Best,
Stavros
Has there been any development with this?
@manzt we have not developed a JS API for TileDB yet. We are still evaluating what would be best for users, and their use cases. Would you mind explaining where you are trying to use TileDB from JS? Is it for Node.js server side? Or are you looking for a way to query TileDB data from a browser client side?
To give some background, we have evaluated making a Node.js package wrapping our C api, and believe this to be doable. This however only benefits server side applications (or things like electron). For client side we’ve considered web assembly, but find there will be constraints to parallelism and memory usage/effective data set sizes since things would be in a browser. With this we’d like to know more about your goals and wishes for a JS package so we can determine what is needed.
On a related note, we have started an experimental API for JavaScript for TileDB Cloud, TileDB-Cloud-JS. To be clear, this package for TileDB Cloud will not replace an eventual JavaScript API for core TileDB. We started the Cloud JS package as we had a user asking about running serverless SQL from the web browser. This package wraps our cloud REST API, which makes it a nice fit into the javascript ecosystem. If you are more interested in client side or the cloud, let us know and I’m happy to discuss the cloud api more, and provide examples as we work to improve this package and its documentation.
1 Like
Hi, I have a possible scenario for TileDB on the client (web browser) side… I have a couple of datasets that need to be analysed and rendered in a web browser. The interesting part is that although the data sets are (or will soon be) open source, the sub-sets of the data (and who might be using the data) could be considered “sensitive”. Sending the data sets (which are currently under 100MB in total as an sqlite db file) to the browser is feasible and doing all the data analysis in the browser keeps all the potentially sensitive stuff in the users browser (thus alleviating many of their concerns). There are 2 parts to the data set, one is effectively a list of information about “assets” (industrial plants of one sort or another) which fits nicely into a small number of relational tables. The other part is effectively a grid of ~25k lat/lon co-ordinates covering the globe, for each point there is a “risk value” calculated for a risk type (such as flooding/sea level rise/water table depletion/sever storm) for a number of points in time (i.e. every 5 years between 2020 and 2100). I receive this data as a netCDF4 file, which is a good fit for this data. Unfortunately I haven’t found a webassembly (or pure javascript) library that can read netCDF within a browser so for now this data is also being shoe-horned into a couple of relational tables. I’m using sql.js (sqlite complied into webassembly) running in a service worker in the browser - I am astonished with how well this works (the data transfer rate out of sqlite to the main browser event thread 
The down side is that adding all the grid data (and all the indexes on that grid data needed to get it back out again) significantly increases the file size of the relational database. I suspect that a webassembly version of TileDB would be an excellent fit for this “grid data”.
Being able to run in the browser would be very useful, as mentioned above. Perhaps this could be done as a plugin. Even better, although it is a longer path, would be to create a localstorage / indexdDB alternative that directly implements TileDB natively, using threads, or perhaps using a separate process with shared memory messaging. Using the same security rules as TileDB, this would be a powerful feature, enabling many kinds of web, mobile, and desktop applications. Local storage areas could be defined by the user, and perhaps also cloud storage could be registered in a similar way. Replica sets would also be a nice addition, where everything gets stored N times synchronously or asynchronously as desired.