---

!!! STOP USING jet-parsing-api clone !!!
It has been merged to trunk.

---

Motivation

NetBeans IDE is no longer just Java IDE. We support many various languages: Java, Ruby, PHP, JSP, Javascript and so on. Each language support module implements its own parsing framework. We have Retouche for Java, GSF for Ruby, Schliemann. Now is the right time to consiladate those framworks in order to avoid code duplication, improve performance, solve threading issues, consolidate features and allow language embedding.

Description

We want to create Parsing API to:

• Unify registration of parsers
• Allow language embedding
• Allow file indexing
• Unify threading, starting/canceling parser tasks
• Avoid code duplication
• Simplify implementation of language supports

Team

• GSF Rewrite (Vita Stejskal)
• Parsing API (Hanz, Tomas Z. /indexing part/)
• Java based on Parsing API (Tomas Z.)
• JSP based on Parsing API (Honza L.)
• Schlieman based on Parsing API (Hanz)
• GSF langugaes rewrite (TBD)

Schedule

• Checkpoint 1 - Merge Parsing API to trunk
  • API Itself
  • Indexing
  • Java Implementation
  • Schliemann implementation
  • JSP (non GSF Language)
  • Parsing API pushed into trunk
  • resources: 2-3 people
• Checkpoint 2 - GSF and javascript rewritten to Parsing API
  • Nov 28
  • GSF rewritten to Parsing API
  • Prototype available on branch
  • resources: 2-3 people
• Checkpoint 3 : all important GSF-based plugins rewritten to Parsing API
  • 6.7 code freeze, "Checkpoint 2" + 8 weeks
  • See the status table below
  • resources: 5 (needs to be discussed with web team)

Current Status

• The jet-parsing-api clone has been merged to trunk!!

Plugin	Owner	QE	Go/No Go	Status
CSS	mfukala	jsedek		Done, css.visual module still disabled, work in progress.
Groovy	phejl	jungi		Done, bugfixing.
GSP	phejl	jungi		Done, bugfixing.
HTML	mfukala	jsedek		Done, bugfixing.
Javascript	mfukala	mschovanek		Done, bugfixing.
JSP	mfukala	jsedek		Done, bugfixing.
PHP	tslota	mikhail.matveev		Done, bugfixing.
Python	?	--		Not migrated.
RHTML	mkrauskopf	mschovanek		Done, bugfixing.
Ruby	mkrauskopf	mschovanek		Done, bugfixing.
YAML	phejl	--		Done, bugfixing.
JavaFX	asotona, manowar			Will be done post 6.7
Ada	Andrea Lucarelli	--		Done, bugfixing. (contrib)
EJS	--	--		Not planned for 6.7 (contrib)
Erlang	Caoyuan Deng	--		Rewrite done (?) -- Status
Fortress	--	--		Not planned for 6.7 (contrib)
Scala	Caoyuan Deng	--		Done, bugfixing. (contrib)
Tcl/Tk	--	--		Not planned for 6.7 (contrib)

Dependencies

Parsing API must be used by GSF, Retouche and Schliemann for 6.7. And ideally by all language support modules in future versions. If Checkpoint 2 slips, there is high risk of not delivering new GSF for 6.7.

Impacts

Architecture

API

We have experience with Retouche, GSF and Schliemann and we want to make an API, which will rise from our experience with those frameworks. The API is not intend to be revolutionary. It is evolutionary development.

The API will have several parts:

1. Parsing
   • Defines registration of parsers
   • Defines threading
   • Concept of Virtual Source (support for language embedding)
   • No support for phases
   • No ordering of parsers
2. Indexing
   • Provides scanning infrastructure
   • Allows registration of Index providers
   • RepositoryUpdater - listening on file and document changes
3. "Generic ClassPath"
   • Provides something like ClassPath to specify source roots (folders, archives)
4. UserAction & Modification tasks
   • Change represented by ModificationResult (in fact textual diff)
   • Need source position translators for embedded languages

Documentation

API must be well documented. Developer documentation is a must.

Migrating GSF-based language plugins to Parsing & Indexing API

Testing

• Unit testing.
• Functional testing through language support modules.

Performance

• Well defined threading and starting/canceling parser tasks could improve performance of code completion and features in various Editors
• Startup scanning will be slower because of scanning and indexing not only Java, but possibly all files

Risks

This task should be completed in early stage of development cycle to allow API users to use it. We expect shorter development phase and longer stabilization phase. It is necessary to complete Checkpoint 2 on time. Otherwise there is a high risk of not delivering new version of GSF for 6.7. It is necessary to rewrite all languages to new version of GSF because of embedding.

Tasks

We have been tracking all tasks related to Parsing API in IssueZilla under Editor / Parsing & Indexing category. Please use the following query to see all unresolved issues sorted by their priority.

The next immediate task is to merge the Parsing API back to the main codeline, after which the stabilization will continue in main. Here is the list of stoppers that need to be resolved before the clone is merged back to main.

Bugs

• List of bugs specific to the jet-parsing-api clone
• List of stoppers that need to be addressed before merging jet-parsing-api to trunk
• List of (trunk) bugs fixed by the parsing API

Additional information

• Send comments to dev@editor.netbeans.org
• Editor Support Changes Proposal
• Parsing API - Use cases, Requirements
• GSF Inception Review
• Parsing API Inception Review
• Parsing API Technical Council Review
• Migrating GSF-based language plugins to Parsing & Indexing API

Source

• Available at http://hg.netbeans.org/main

Builds

Available at http://deadlock.netbeans.org/hudson/job/trunk/

Attachments