This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
|
teaching:mfe:is [2015/04/13 14:42] svsummer [Engineering a runtime system and compiler for AQL] |
teaching:mfe:is [2015/04/13 14:46] svsummer [Design and Implementation of a Curriculum Revision Tool] |
||
|---|---|---|---|
| Line 67: | Line 67: | ||
| * create associated tooling for SCULPT (i.e., parser and serializer generator, in the spirit of data description tools) | * create associated tooling for SCULPT (i.e., parser and serializer generator, in the spirit of data description tools) | ||
| - | + | \\ | |
| **Deliverables** of this master thesis project: | **Deliverables** of this master thesis project: | ||
| - detailed description of the SCULPT proposal (document) | - detailed description of the SCULPT proposal (document) | ||
| Line 83: | Line 82: | ||
| ===== Engineering a runtime system and compiler for AQL ===== | ===== Engineering a runtime system and compiler for AQL ===== | ||
| - | Automatically extracting structured information from text is a task that has been pursued for decades. As a discipline, ///Information Extraction/// (IE) had its start with the [[http://acl.ldc.upenn.edu/C/C96/C96-1079.pdf|DARPA Message Understanding Conference in 1987]]. While early work in the area focused largely on military applications, recent changes have made information extraction increasingly important to an increasingly broad audience. Trends such as the rise of social media have produced huge amounts of text data, while analytics platforms like Hadoop have at the same time made the analysis of this data more accessible to a broad range of users. Since most analytics over text involves information extraction as a first step, IE is a very important part of data analysis in the enterprise today. | + | Automatically extracting structured information from text is a task that has been pursued for decades.Since most analytics over text involves information extraction as a first step, IE is a very important part of data analysis in the enterprise today. |
| In 2005, researchers at the IBM Almaden Research Center developped a new system specifically geared for practical information extraction in the enterprise. This effort lead to SystemT, a rule-based IE system with an SQL-like declarative language named AQL (Annotation Query Language). The declarative nature of AQL enables new kinds of tools for extractor development, and draws upon known techniques form query processing in relational database management systems to offer a cost-based optimizer that ensures high-througput performance. Recent research into the foundations of AQL (http://researcher.watson.ibm.com/researcher/files/us-fagin/jacm15.pdf) has shown that, as an alternative, it is also possible to build a runtime system for AQL based on special kinds of finite state automata. A potential benefit of this alternate runtime system is that text files need only be processed once (instead of multiple times in the cost-based optimizer backend) and may hence provide greater throughput. On the other hand, the alternate system can sometimes have larger memory requirements than the cost-based optimizer backend. | In 2005, researchers at the IBM Almaden Research Center developped a new system specifically geared for practical information extraction in the enterprise. This effort lead to SystemT, a rule-based IE system with an SQL-like declarative language named AQL (Annotation Query Language). The declarative nature of AQL enables new kinds of tools for extractor development, and draws upon known techniques form query processing in relational database management systems to offer a cost-based optimizer that ensures high-througput performance. Recent research into the foundations of AQL (http://researcher.watson.ibm.com/researcher/files/us-fagin/jacm15.pdf) has shown that, as an alternative, it is also possible to build a runtime system for AQL based on special kinds of finite state automata. A potential benefit of this alternate runtime system is that text files need only be processed once (instead of multiple times in the cost-based optimizer backend) and may hence provide greater throughput. On the other hand, the alternate system can sometimes have larger memory requirements than the cost-based optimizer backend. | ||
| Line 105: | Line 104: | ||
| - An experimental validation of the compiler, comparing efficiency of compiled queries against the cost-based reference compiler. | - An experimental validation of the compiler, comparing efficiency of compiled queries against the cost-based reference compiler. | ||
| - | + | \\ | |
| - | ---- | + | |
| **References about SystemT**: | **References about SystemT**: | ||
| * [[http://almaden.ibm.com/cs/projects/avatar/icde2008.pdf|An Algebraic Approach to Rule-Based Information Extraction]] | * [[http://almaden.ibm.com/cs/projects/avatar/icde2008.pdf|An Algebraic Approach to Rule-Based Information Extraction]] | ||
| * [[http://www.sigmod.org/publications/sigmod-record/0812/p007.special.krishnamurthy.pdf|SystemT: A System for Declarative Information Extraction]] | * [[http://www.sigmod.org/publications/sigmod-record/0812/p007.special.krishnamurthy.pdf|SystemT: A System for Declarative Information Extraction]] | ||
| + | \\ | ||
| **References about finite state automata evaluation**: | **References about finite state automata evaluation**: | ||
| * Regular expression pattern matching can be simple and fast. http://swtch.com/~rsc/regexp/regexp1.html | * Regular expression pattern matching can be simple and fast. http://swtch.com/~rsc/regexp/regexp1.html | ||
| Line 118: | Line 116: | ||
| * [[http://www.diku.dk/kmc/documents/AiPL-CrashCourse.pdf|A Crash-Course in Regular Expression Parsing and Regular Expressions as Types.]] | * [[http://www.diku.dk/kmc/documents/AiPL-CrashCourse.pdf|A Crash-Course in Regular Expression Parsing and Regular Expressions as Types.]] | ||
| + | \\ | ||
| **Interested?** Contact : [[stijn.vansummeren@ulb.ac.be|Stijn Vansummeren]] | **Interested?** Contact : [[stijn.vansummeren@ulb.ac.be|Stijn Vansummeren]] | ||
| + | \\ | ||
| **Status**: available | **Status**: available | ||
| Line 144: | Line 144: | ||
| Datalog is a fundamental query language in datamanagement based on logic programming. It essentially extends select-from-where SQL queries with recursion. There is a recent trend in data management research to use datalog to specify distributed applications, most notably on the web, as well as do inference on the semantic web. The goal of this thesis is to engineer a basic **distributed datalog system**, i.e., a system that is capable of compiling & running distributed datalog queries. The system should be implemented in the Scala programming language. Learning Scala is part of the master thesis project. | Datalog is a fundamental query language in datamanagement based on logic programming. It essentially extends select-from-where SQL queries with recursion. There is a recent trend in data management research to use datalog to specify distributed applications, most notably on the web, as well as do inference on the semantic web. The goal of this thesis is to engineer a basic **distributed datalog system**, i.e., a system that is capable of compiling & running distributed datalog queries. The system should be implemented in the Scala programming language. Learning Scala is part of the master thesis project. | ||
| - | The system should: | + | The system should incorporate recently proposed worst-case join algorithms (i.e., the [[http://arxiv.org/abs/1210.0481|leapfrog trie join]]) and employ known local datalog optimizations (such as magic sets and QSQ.) |
| - | * incorporate recently proposed worst-case join algorithms (i.e., the [[http://arxiv.org/abs/1210.0481|leapfrog trie join]]) | + | |
| - | * employ known local datalog optimizations (such as magic sets and QSQ) | + | |
| **Validation of the approach** The thesis should propose a benchmark collection of datalog queries and associated data workloads that be used to test the obtained system, and measure key performance characteristics (elasticity of the system; memory frootprint; overall running time, ...) | **Validation of the approach** The thesis should propose a benchmark collection of datalog queries and associated data workloads that be used to test the obtained system, and measure key performance characteristics (elasticity of the system; memory frootprint; overall running time, ...) | ||
| - | **Deliverables**: | ||
| + | **Deliverables**: | ||
| * Semantics of datalog; overview of known optimization strategies (document) | * Semantics of datalog; overview of known optimization strategies (document) | ||
| * Description of the leapfrog trie join (document) | * Description of the leapfrog trie join (document) | ||
| Line 157: | Line 155: | ||
| * Experimental analysis of developped system on a number of use cases (document) | * Experimental analysis of developped system on a number of use cases (document) | ||
| + | \\ | ||
| **Interested?** Contact : [[stijn.vansummeren@ulb.ac.be|Stijn Vansummeren]] | **Interested?** Contact : [[stijn.vansummeren@ulb.ac.be|Stijn Vansummeren]] | ||
| Line 181: | Line 180: | ||
| * Interact with the administration of the Ecole Polytechnique to fine-tune the above requirements; test the implementation; and integrate remarks after testing | * Interact with the administration of the Ecole Polytechnique to fine-tune the above requirements; test the implementation; and integrate remarks after testing | ||
| + | \\ | ||
| **Interested?** Contact : Stijn Vansummeren (stijn.vansummeren@ulb.ac.be), Frédéric Robert <frrobert@ulb.ac.be> | **Interested?** Contact : Stijn Vansummeren (stijn.vansummeren@ulb.ac.be), Frédéric Robert <frrobert@ulb.ac.be> | ||
| Line 269: | Line 269: | ||
| Interested? Contact [[toon.calders@ulb.ac.be|Toon Calders]] | Interested? Contact [[toon.calders@ulb.ac.be|Toon Calders]] | ||
| - | |||
| - | ===== Design and Implementation of a Curriculum Revision Tool ====== | ||
| - | |||
| - | Stijn Vansummeren (WIT), Frédéric Robert (BEAMS) | ||
| - | |||
| - | This MFE concers the analysis, design, and implementation of a | ||
| - | software system that can assist in the revision of teaching curricula | ||
| - | (also known as teaching programs). | ||
| - | |||
| - | The primary targetted functionalities of the software system are as | ||
| - | follows: | ||
| - | * It should allow to make different versions of the teaching programs, much in the same way as version control systems like GIT and subversion offer the possibility to make different "development branches" of a program's source code. | ||
| - | * It should allow an extensible means to check the modified program for inconsistentcies. (For example, if course X has course Y as prerequisite, then course Y should not be scheduled in 2nd semester and X in 1st semester. Moreover, the total number of ECTS of all courses should be at most 60 ECTS. ) | ||
| - | * It should allow to analyze the modifications proposed in the teaching programs, and summarize the impact that these changes could have on other programs. (For example, if a course is removed from the computer science curriculum, it should be flagged that it should also be removed from all curricula that included the course.) | ||
| - | * It should load data from (and preferably, save data to) the ULB central administration database. | ||
| - | * It should give suggestions concerning the impact of the modifications on the course schedules. | ||
| - | |||
| - | A proof-of-concept implementation of a revision tool that supports the first two requirements above is currently being developed in the context of a PROJH402 project. The MFE student that selects this topic is expected to: | ||
| - | |||
| - | * Develop this prototype to a production-ready implementation. | ||
| - | * Implement the communication with the central ULB database. | ||
| - | * Implement the impact analysis concerning the course schedules. | ||
| - | * Interact with the administration of the Ecole Polytechnique to fine-tune the above requirements; test the implementation; and integrate remarks after testing | ||
| - | |||
| - | Contact : Stijn Vansummeren <stijn.vansummeren@ulb.ac.be>, Frédéric Robert <frrobert@ulb.ac.be> | ||