This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
|
teaching:mfe:is [2015/04/13 14:26] svsummer |
teaching:mfe:is [2015/04/13 14:44] svsummer [Engineering a runtime system and compiler for AQL] |
||
|---|---|---|---|
| Line 24: | Line 24: | ||
| * Contact : [[ezimanyi@ulb.ac.be|Esteban Zimanyi]] | * Contact : [[ezimanyi@ulb.ac.be|Esteban Zimanyi]] | ||
| - | |||
| ===== Compiling SPARQL queries into machine code ===== | ===== Compiling SPARQL queries into machine code ===== | ||
| Line 30: | Line 29: | ||
| Due to the increasing availability of larger and larger cheap RAM memories, the working set of modern database management systems becomes more and more main memory resident. This implies that, in contrast to traditional database management systems, slow disk accesses are rare, and that hence, the in-memory processing speed of databases becomes an important factor. As recently observed by a number of researchers, (e.g., [[http://sites.computer.org/debull/A14mar/p3.pdf|Neumann and Leis]]), one very attractive approach for fast query processing in this context is the just-in-time compilation of incoming queries into machine code. This compilation avoids the overhead of the traditional interpretation of query plans, and can aid in minimzing memory traffic for boosting performance. | Due to the increasing availability of larger and larger cheap RAM memories, the working set of modern database management systems becomes more and more main memory resident. This implies that, in contrast to traditional database management systems, slow disk accesses are rare, and that hence, the in-memory processing speed of databases becomes an important factor. As recently observed by a number of researchers, (e.g., [[http://sites.computer.org/debull/A14mar/p3.pdf|Neumann and Leis]]), one very attractive approach for fast query processing in this context is the just-in-time compilation of incoming queries into machine code. This compilation avoids the overhead of the traditional interpretation of query plans, and can aid in minimzing memory traffic for boosting performance. | ||
| - | A number of recent research prototypes exist that compile SQL queries into machine code in this sense: | + | A number of recent research prototypes exist that compile SQL queries into machine code in this sense: HyPer A Hybrid OLTP&OLAP High Performance DBMS (http://hyper-db.de/) and Legobase (https://github.com/epfldata/NewLegoBase and http://data.epfl.ch/legobase). |
| - | * HyPer – A Hybrid OLTP&OLAP High Performance DBMS http://hyper-db.de/ | + | |
| - | * Legobase - https://github.com/epfldata/NewLegoBase and http://data.epfl.ch/legobase | + | |
| The objective of this master thesis is to apply the same methodology to engineer a compiler that translates (fragments of) SPARQL (the standard query language for querying RDF data on the semantic web) into machine code. The overall methodology should follow the methodology used by HyPer and Legobase: | The objective of this master thesis is to apply the same methodology to engineer a compiler that translates (fragments of) SPARQL (the standard query language for querying RDF data on the semantic web) into machine code. The overall methodology should follow the methodology used by HyPer and Legobase: | ||
| Line 43: | Line 40: | ||
| **Validation of the approach** The thesis should propose a benchmark collection of SPARQL queries that can be used to test the obtained SPARQL-to-machine-code compiler and compare its perforance against a reference, interpreter-based SPARQL compiler. | **Validation of the approach** The thesis should propose a benchmark collection of SPARQL queries that can be used to test the obtained SPARQL-to-machine-code compiler and compare its perforance against a reference, interpreter-based SPARQL compiler. | ||
| - | **Deliverables** of the master thesis project | + | **Deliverables** of the master thesis project: |
| - | * An overview of the state of the art in query-to-machine-code compilation. | + | - An overview of the state of the art in query-to-machine-code compilation. |
| - | * A description of latent modular staging and how it can be used to construct machine-code compilers. | + | - A description of latent modular staging and how it can be used to construct machine-code compilers. |
| - | * The SPARQL compiler (software artifact) | + | - The SPARQL compiler (software artifact) |
| - | * A benchmark set of SPARQL queries and associated data sets for the experimental validation | + | - A benchmark set of SPARQL queries and associated data sets for the experimental validation |
| - | * An experimental validation of the compiler, comparing efficiency of compiled queries against a reference compiler based on query plan interpretation. | + | - An experimental validation of the compiler, comparing efficiency of compiled queries against a reference compiler based on query plan interpretation. |
| - | **Interested?** | + | |
| - | * Contact : //Stijn Vansummeren// | + | **Interested?** Contact : [[stijn.vansummeren@ulb.ac.be|Stijn Vansummeren]] |
| **Status**: available | **Status**: available | ||
| Line 64: | Line 61: | ||
| The objective of this master thesis is to implement a recent proposal for such a schema language named SCULPT (http://arxiv.org/abs/1411.2351). Concretely, this entails: | The objective of this master thesis is to implement a recent proposal for such a schema language named SCULPT (http://arxiv.org/abs/1411.2351). Concretely, this entails: | ||
| - | + | * proposing an elegant concrete syntax for SCULPT schemas | |
| - | * propose an elegant concrete syntax for SCULPT schemas | + | |
| * implement both the in-memory and streaming validation algorithms of SCULPT proposed in http://arxiv.org/abs/1411.2351 | * implement both the in-memory and streaming validation algorithms of SCULPT proposed in http://arxiv.org/abs/1411.2351 | ||
| * extend the SCULPT proposal, by investigating how SCULPT can be combined with complementary features recently proposed by the W3C CSV on the Web Working group (http://www.w3.org/2013/csvw/wiki/Main_Page) | * extend the SCULPT proposal, by investigating how SCULPT can be combined with complementary features recently proposed by the W3C CSV on the Web Working group (http://www.w3.org/2013/csvw/wiki/Main_Page) | ||
| * and in particular, extend sculpt with features that allow tabular files to be converted into RDF | * and in particular, extend sculpt with features that allow tabular files to be converted into RDF | ||
| * create associated tooling for SCULPT (i.e., parser and serializer generator, in the spirit of data description tools) | * create associated tooling for SCULPT (i.e., parser and serializer generator, in the spirit of data description tools) | ||
| + | |||
| + | |||
| **Deliverables** of this master thesis project: | **Deliverables** of this master thesis project: | ||
| - | * detailed description of the SCULPT proposal (document) | + | - detailed description of the SCULPT proposal (document) |
| - | * overview of the state of the art; in particular other proposals for schema languages for tabular data (document) | + | - overview of the state of the art; in particular other proposals for schema languages for tabular data (document) |
| - | * concrete syntax for sculpt (design document + formal grammar) | + | - concrete syntax for sculpt (design document + formal grammar) |
| - | * implementation of SCULPT validation algorithms (software artifact) | + | - implementation of SCULPT validation algorithms (software artifact) |
| - | * extension of sculpt with features for converting into RDF (document + software) | + | - extension of sculpt with features for converting into RDF (document + software) |
| - | **Interested?** | + | **Interested?** Contact: [[stijn.vansummeren@ulb.ac.be|Stijn Vansummeren]] |
| - | * Contact: //Stijn Vansummeren// | + | |
| **Status**: available | **Status**: available | ||
| Line 87: | Line 85: | ||
| Automatically extracting structured information from text is a task that has been pursued for decades. As a discipline, ///Information Extraction/// (IE) had its start with the [[http://acl.ldc.upenn.edu/C/C96/C96-1079.pdf|DARPA Message Understanding Conference in 1987]]. While early work in the area focused largely on military applications, recent changes have made information extraction increasingly important to an increasingly broad audience. Trends such as the rise of social media have produced huge amounts of text data, while analytics platforms like Hadoop have at the same time made the analysis of this data more accessible to a broad range of users. Since most analytics over text involves information extraction as a first step, IE is a very important part of data analysis in the enterprise today. | Automatically extracting structured information from text is a task that has been pursued for decades. As a discipline, ///Information Extraction/// (IE) had its start with the [[http://acl.ldc.upenn.edu/C/C96/C96-1079.pdf|DARPA Message Understanding Conference in 1987]]. While early work in the area focused largely on military applications, recent changes have made information extraction increasingly important to an increasingly broad audience. Trends such as the rise of social media have produced huge amounts of text data, while analytics platforms like Hadoop have at the same time made the analysis of this data more accessible to a broad range of users. Since most analytics over text involves information extraction as a first step, IE is a very important part of data analysis in the enterprise today. | ||
| - | Broadly speaking, there are two main schools of thought on the realization of IE: the **statistical** (machine-learning) methodology and the **rule-based approach**. Both categories of systems have their strenght and weaknesses. One of the most notable weaknesses is performance: tranditional rule-based systems, for example were mainly built for academic settings, where extraction throughput is far less important than the accuracy of results. | + | In 2005, researchers at the IBM Almaden Research Center developped a new system specifically geared for practical information extraction in the enterprise. This effort lead to SystemT, a rule-based IE system with an SQL-like declarative language named AQL (Annotation Query Language). The declarative nature of AQL enables new kinds of tools for extractor development, and draws upon known techniques form query processing in relational database management systems to offer a cost-based optimizer that ensures high-througput performance. Recent research into the foundations of AQL (http://researcher.watson.ibm.com/researcher/files/us-fagin/jacm15.pdf) has shown that, as an alternative, it is also possible to build a runtime system for AQL based on special kinds of finite state automata. A potential benefit of this alternate runtime system is that text files need only be processed once (instead of multiple times in the cost-based optimizer backend) and may hence provide greater throughput. On the other hand, the alternate system can sometimes have larger memory requirements than the cost-based optimizer backend. |
| - | + | ||
| - | In 2005, researchers at the IBM Almaden Research Center developped a new system specifically geared for practical information extraction in the enterprise. This effort lead to SystemT, a rule-based IE system with an SQL-like declarative language named AQL (Annotation Query Language). The declarative nature of AQL enables new kinds of tools for extractor development, and draws upon known techniques form query processing in relational database management systems to offer a cost-based optimizer that ensures high-througput performance. | + | |
| - | + | ||
| - | Recent research into the foundations of AQL (http://researcher.watson.ibm.com/researcher/files/us-fagin/jacm15.pdf) has shown that, as an alternative, it is also possible to build a runtime system for AQL based on special kinds of finite state automata. A potential benefit of this alternate runtime system is that text files need only be processed once (instead of multiple times in the cost-based optimizer backend) and may hence provide greater throughput. On the other hand, the alternate system can sometimes have larger memory requirements than the cost-based optimizer backend. | + | |
| The objective of this master thesis is to design and engineer a runtime system and compiler for (a fragment) of AQL based on finite state automata. Ideally, to obtain the best performance, these automata should be compiled into machine-code when executed. For this compilation, the following technologies should be used: | The objective of this master thesis is to design and engineer a runtime system and compiler for (a fragment) of AQL based on finite state automata. Ideally, to obtain the best performance, these automata should be compiled into machine-code when executed. For this compilation, the following technologies should be used: | ||
| Line 103: | Line 97: | ||
| **Deliverables** of the master thesis project: | **Deliverables** of the master thesis project: | ||
| - | * An overview of AQL, SystemT, and its cost-based optimizer and evaluation engine. (document) | + | - An overview of AQL, SystemT, and its cost-based optimizer and evaluation engine. (document) |
| - | * A description of how AQL can be evaluated by means of so-called vset finite state automata. (document) | + | - A description of how AQL can be evaluated by means of so-called vset finite state automata. (document) |
| - | * A detailed desription of the state of the art in evaluating finite state automata. (document) | + | - A detailed desription of the state of the art in evaluating finite state automata. (document) |
| - | * Identification of the AQL syntaxt that is to be supported. (specification) | + | - Identification of the AQL syntaxt that is to be supported. (specification) |
| - | * The AQL compiler (software artifact) | + | - The AQL compiler (software artifact) |
| - | * A benchmark set of AQL queries and associated data sets for the experimental validation | + | - A benchmark set of AQL queries and associated data sets for the experimental validation |
| - | * An experimental validation of the compiler, comparing efficiency of compiled queries against the cost-based reference compiler. | + | - An experimental validation of the compiler, comparing efficiency of compiled queries against the cost-based reference compiler. |
| - | **References about SystemT**: -[[http://almaden.ibm.com/cs/projects/avatar/icde2008.pdf|An Algebraic Approach to Rule-Based Information Extraction]] -[[http://www.sigmod.org/publications/sigmod-record/0812/p007.special.krishnamurthy.pdf|SystemT: A System for Declarative Information Extraction]] | + | \\ |
| + | **References about SystemT**: | ||
| + | * [[http://almaden.ibm.com/cs/projects/avatar/icde2008.pdf|An Algebraic Approach to Rule-Based Information Extraction]] | ||
| + | * [[http://www.sigmod.org/publications/sigmod-record/0812/p007.special.krishnamurthy.pdf|SystemT: A System for Declarative Information Extraction]] | ||
| + | \\ | ||
| **References about finite state automata evaluation**: | **References about finite state automata evaluation**: | ||
| * Regular expression pattern matching can be simple and fast. http://swtch.com/~rsc/regexp/regexp1.html | * Regular expression pattern matching can be simple and fast. http://swtch.com/~rsc/regexp/regexp1.html | ||
| Line 119: | Line 117: | ||
| * [[http://www.diku.dk/kmc/documents/AiPL-CrashCourse.pdf|A Crash-Course in Regular Expression Parsing and Regular Expressions as Types.]] | * [[http://www.diku.dk/kmc/documents/AiPL-CrashCourse.pdf|A Crash-Course in Regular Expression Parsing and Regular Expressions as Types.]] | ||
| - | **Interested?** | + | **Interested?** Contact : [[stijn.vansummeren@ulb.ac.be|Stijn Vansummeren]] |
| - | * Contact : //Stijn Vansummeren// | + | |
| **Status**: available | **Status**: available | ||
| + | |||
| ===== Structural compression of relational databases ===== | ===== Structural compression of relational databases ===== | ||
| Line 129: | Line 127: | ||
| The goal of this thesis is to study scalable algorithms for constructing the actual succinct representations. Some in-memory algorithms are already known, but given the large size of typical database, distributed and out-of-core alternatives need to be found. | The goal of this thesis is to study scalable algorithms for constructing the actual succinct representations. Some in-memory algorithms are already known, but given the large size of typical database, distributed and out-of-core alternatives need to be found. | ||
| - | |||
| - | **Validation**: apply the compression algorithm on a number of relational databases; illustrate the achieved compression as well as performance characterstics of the developped algorithm (out-of-core characterstics; scaleup; ...). | ||
| **Deliverables**: | **Deliverables**: | ||
| Line 139: | Line 135: | ||
| * Experimental analysis of distributed algorithm on a number of datasets. (document) | * Experimental analysis of distributed algorithm on a number of datasets. (document) | ||
| - | **Required reading**: | + | **Interested?** Contact : [[stijn.vansummeren@ulb.ac.be|Stijn Vansummeren]] |
| - | * Computing simulations on finite and infinite graphs. Henzinger, henzinger and Kopke | + | |
| - | * Ranzato and Tapparo - An efficient simulation algorithm based on abstract interpretation | + | |
| - | * Ranzato - An efficient simulation algorithm on Kripke structures. | + | |
| - | * Blom, Orzan - Distributed State Space Minimization | + | |
| - | * Blom, Orzan - A distributed algorithm for strong bisimulation reduction of state spaces | + | |
| - | * Ma et al - distributed graph pattern matching | + | |
| - | * (+ signature-based simulation part of Yongming Luo's thesis). | + | |
| - | + | ||
| - | **Interested?** | + | |
| - | * Contact : //Stijn Vansummeren// | + | |
| **Status**: available | **Status**: available | ||
| Line 162: | Line 148: | ||
| **Validation of the approach** The thesis should propose a benchmark collection of datalog queries and associated data workloads that be used to test the obtained system, and measure key performance characteristics (elasticity of the system; memory frootprint; overall running time, ...) | **Validation of the approach** The thesis should propose a benchmark collection of datalog queries and associated data workloads that be used to test the obtained system, and measure key performance characteristics (elasticity of the system; memory frootprint; overall running time, ...) | ||
| - | |||
| - | **Required reading**: | ||
| - | * Datalog and Recursive Query Processing - Foundations and trends in query processing. | ||
| - | * LogicBlox, Platform and Language: A Tutorial (Todd J. Green, Molham Aref, and Grigoris Karvounarakis) | ||
| - | * Dedalus: Datalog in Time and Space (Peter Alvaro, William R. Marczak, Neil Conway, Joseph M. Hellerstein, David Maier, and Russell Sears) | ||
| - | * Declarative Networking (Loo et al). For the distributed evaluation strategy. | ||
| - | * Parallel processing of recursive queries in distributed architectures (VLDB 1989) | ||
| - | * Evaluating recursive queries in distributed databases (IEEE trans knowledge and data engieneering, 1993) | ||
| **Deliverables**: | **Deliverables**: | ||
| + | |||
| * Semantics of datalog; overview of known optimization strategies (document) | * Semantics of datalog; overview of known optimization strategies (document) | ||
| - | * Description of the leapfrog trie join | + | * Description of the leapfrog trie join (document) |
| + | * Datalog system (software artifact) | ||
| + | * Experimental analysis of developped system on a number of use cases (document) | ||
| - | **Interested?** | + | **Interested?** Contact : [[stijn.vansummeren@ulb.ac.be|Stijn Vansummeren]] |
| - | * Contact : //Stijn Vansummeren// | + | |
| **Status**: available | **Status**: available | ||
| Line 195: | Line 175: | ||
| A proof-of-concept implementation of a revision tool that supports the first two requirements above is currently being developped in the context of a PROJH402 project. The MFE student that selects this topic is expected to: | A proof-of-concept implementation of a revision tool that supports the first two requirements above is currently being developped in the context of a PROJH402 project. The MFE student that selects this topic is expected to: | ||
| - | |||
| * Develop this prototype to a production-ready implementation. | * Develop this prototype to a production-ready implementation. | ||
| * Implement the communication with the central ULB database. | * Implement the communication with the central ULB database. | ||
| Line 201: | Line 180: | ||
| * Interact with the administration of the Ecole Polytechnique to fine-tune the above requirements; test the implementation; and integrate remarks after testing | * Interact with the administration of the Ecole Polytechnique to fine-tune the above requirements; test the implementation; and integrate remarks after testing | ||
| - | **Interested?** | + | **Interested?** Contact : Stijn Vansummeren (stijn.vansummeren@ulb.ac.be), Frédéric Robert <frrobert@ulb.ac.be> |
| - | * Contact : Stijn Vansummeren (stijn.vansummeren@ulb.ac.be), Frédéric Robert <frrobert@ulb.ac.be> | + | |