logoalt Hacker News

sixdimensionalyesterday at 11:33 PM1 replyview on HN

In the past platforms have integrated ML algorithms into relational databases and SQL through extensions (both commercial and open source). A famous open source one was MADlib [1], which has an implementation of neural networks. Even the commercial ones were similar, I used ML algorithms in SQL Server many years ago around 2005 I think.

I am wondering about.. SQL as a declarative structured query language that can be optimized into essentially any kind of distributed, directed acyclic graph of processing - vs the special characteristics of relational databases (relational algebra, relvars, etc. etc.) is an important distinction as - of yet, I see the author linking both together so I'm trying to understand what it is about relational structures that specifically helped here (just not seeing it yet, not that it isn't there).

Also, wondering if ISO/IEC 9075-15:2023 SQL standard for multidimensional arrays (MDA) is of any use here? Paper describing SQL/MDA here [2].

[1] https://madlib.apache.org/documentation.html

[2] https://www.ifis.uni-luebeck.de/~moeller/Lectures/WS-19-20/N...


Replies

alxmrsyesterday at 11:50 PM

I need to better understand your first question before I can comment. In theory, we could work with MADlib too -- what we do is port scientific data, which typically is a "tensor" or Nd array, into a tabular view. I believe you know relational theory better than I do, I am still fairly new to the field.

WRT ISO/IEC 9075-15:2023: This is the standard established from rasdaman, IIUC. I reject this approach (which treats arrays as a column type), and instead adopt one inspired by Michael Stonebraker's SciDB (which treats arrays as tables themselves). For an in depth review of the topic, I recommend this NSF paper: https://par.nsf.gov/servlets/purl/10545560