LexerMigrationJ2EE

Contents


Migration of J2EE editors to Lexer

One of the primary goals of the Netbeans 6.0 release is scripting support. Since the scripting support is largely based on editing and most of the scripting languages planned to be supported (JavaScript, Phobos Embedded JavaScript, Ruby on Rails) somehow encompasses HTML, we need to adapt our existing code to cover the new usecases.

Migration to lexer needs to be done for following editors held by j2ee team:

  • X/HTML - acting as a content language in other scripting languages (JSP, EJS, ROR). Can also contain scripting language e.g. javascript,
  • JSP(+EL) - not directly connected to other scripting languages (all interaction is done through HTML), but tightly coupled with HTML so when we migrate HTML, we need to migrate JSP as well,
  • XML/DTD - migration is not that important from the embedding point of view (we can workaround using XML as content language in JSP documents by using HTML syntax),
  • CSS - needs to be migrated since is heavily interacts with HTML/XML.

What needs to be done for lexerization of the current J2EE editors

Basically we need to switch all current syntax classes (extending org.netbeans.editor.Syntax class) to implement the lexer api (=org.netbeans.spi.lexer.Lexer=). Then we need to migrate all features based on top of the old syntaxes to use the new lexer API. The lexer api is pretty straightforward so the changes are more or less mechanical, though there are some exceptions where the migration involves reconsideration of the semantic like migration of JSPMultiSyntax & JSPTagSyntax to JSPLexer or some old-syntax hacks for example in JSPSyntaxSupport.

Following list contains areas/classes which needs to be adapted to work withh lexer:

HTML

  • create a new html/lexer module with HTMLLexer and other Lexer's SPI classes,
  • update coloring information for tokens (token names)
  • update the impl. of NbEditorKit class to enable the lexer coloring (put LexerLayer into the UI layers)
  • completion provider related stuff (HTMLCompletionQuery mainly)
  • fold manager (HTMLFoldManager)
  • formatter engine (HTMLFormatter) + we also need to migrate its base TagBasedFormatter which is heavily using old syntaxes
  • HTMLSyntaxSupport is mostly about:
  • findMatchingBlock()
  • syntax elements support (a base for completion and folding for example).

CSS

  • move the CSS editor from xml/css to either html/css or web/css module (we might not maintain the xml support for 6.0???)
  • create handcoded CSS lexer or adapt the .jj (javacc) definitions to generate lexer compliant code

The rest of features (check & copy style actions) are syntax implementation independent.

JSP

  • create a new web/jspsyntax/lexer module with JSPLexer and other Lexer's SPI classes. This item requires merging functionality of JspMultiSyntax and JspTagSyntax including cooperation of the lexer syntax with parser (update coloring based on jasper parse data)
  • update coloring information for tokens (token names)
  • update the impl. of NbEditorKit class to enable the lexer coloring (put LexerLayer into the UI layers)
  • bracket matching action in JSPKit
  • completion provider related stuff (JspCompletionQuery mainly)
  • fold manager (JspFoldManager)
  • hyperlinking provider (JSPHyperLinkProvider)
  • JSP/java formatter engines (mostly JspFormatter ) + we also need to migrate its base TagBasedFormatter which is heavily using old syntaxes
  • JSPJavaSyntaxSupport - no need to migrate, will be deleted or reimplemented as a part of Retouche adoption in JSP completion
  • error highlighting (JspParserErrorAnnotation)
  • JspSyntaxSupport is mostly about:
  • findMatchingBlock()
  • syntax elements support (a base for completion and folding for example).
  • some utility methods

Expression language (EL)

This syntax is not used standalone, but just as a part of JSP file.

  • create a new web/el/lexer module with ELLexer and other Lexer's SPI classes. The motivation for having a separate EL module is that there might be its separate usage. See Jakarta Commons EL and Jakarta JEXL.
  • update coloring information for tokens (token names)
  • completion provider related stuff (JspCompletionProvider and ELExpression mainly)

XML

  • create a new xml/lexer module with XMLLexer and other Lexer's SPI classes.
  • update coloring information for tokens (token names)
  • update the impl. of NbEditorKit class to enable the lexer coloring (put LexerLayer into the UI layers)
  • code completion XMLCompletionProvider, SyntaxQueryHelper, heavily depeneds on syntax elements
  • xml comment/uncomment actions in XMLKit
  • NOT fold manager XmlFoldManager depends just on syntax elements
  • navigator - DocumentModelProvider implementation (XMLDocumentModelProvider) used by XML navigator needs to be adapted. The navigator itself is independent.
  • formatter engine (XMLFormatter ) + we also need to migrate its base TagBasedFormatter which is heavily using old syntaxes
  • org.netbeans.modules.xml.text.syntax.dom package needs to be migrated. Many of the DOM impl. classes uses syntax layer.
  • XMLSyntaxSupport is mostly about:
  • findMatchingBlock()
  • syntax elements support (a base for completion and folding for example).

We also exposes the syntax classes as a friend API to these modules:

  • org.netbeans.modules.mobility.end2end
  • org.netbeans.modules.web.core.syntax
  • org.netbeans.modules.xml.retriever

so we alse need to adapt/notify them.

Enterprise pack modules like xml/xdm (XML Document Model) having implementation dependency to xml/text-edit also needs to be migrated, or we can let them use the old syntax layer.

DTD

Similar situation as CSS - we have javacc definition for DTD syntax so we generated the syntax scanner. First option is to handcode DTD lexer or migrate all the infrastructure in org.netbeans.modules.xml.text.syntax.javacc(.lib)

 * formatter engine (DTDFormatter)

Other scripting related changes

The previous chapter described changes required *just to get what we have now* using lexer. There will be other changes required to be done related to the scripting/lexer:

  • reflect the embedding usecases in the features - this means that for example the completion provider has to count with the case that its language doesn't have to be the top-level one, but it can also be nested. So the code has to verify that and eventiually do some other necessary steps.
  • allow to interact with Schlieman - since this is not discovered yet, I am just guessing that there might be some required changes in our code to enable the schlieman features in milti-language environment.
  • exploit Schlieman - if Schlieman exposes its higher-level features (parser, AST) we will likely use it to built our higl level features like navigator, error highlighting etc.
  • ...
Not logged in. Log in, Register

By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo