How to create support for a new language

New Language Support Tutorial

This tutorial is out of date and parts of it are wrong and/or obsolete.

The replacement tutorials for all the text that follows below are:

* NetBeans JavaCC Lexer Tutorial http://platform.netbeans.org/tutorials/nbm-javacc-lexer.html

* NetBeans JavaCC Parser Tutorial http://platform.netbeans.org/tutorials/nbm-javacc-parser.html

1) Overview

This tutorial aims to help NetBeans Platform developers implement support for a new language in their application, for example, in NetBeans IDE. It is focused on editor features such as syntax coloring, error highlighting, parsing, code folding, code completion, etc. Descriptions of how to implement a new project or to provide "run" or "debug" support are out of the scope of this document.

This tutorial contains links to detailed API descriptions for APIs used during language support implementation and provides links to modules you will need to depend on. It is based on NetBeans Platform 6.7 or above (e.g., 6.8 too). Previous versions of the NetBeans Platform do not contain all the APIs used here and there are several new ways of implementing the features relevant to this document.

We will use the "javaCC" parser generator in this tutorial (go here for ANTLR), so you will learn some basic info about lexers, grammars, error recovery and the javaCC parser generator. Creating a new language, and defining its grammar, is not an easy task. That is why we will use an existing Java 1.5 grammar file in this tutorial.

You are assumed to have some basic knowledge of lexical analysis, syntactical analysis, and a basic understanding of language grammars.

Note: If you do not have any/much experience with NetBeans module development, you are highly recommended to follow several tutorials on the NetBeans Platform Learning Trail BEFORE continuing with the steps that follow. In particular, the File Type Integration Tutorial should be very familiar to you.

The sources of the module that you build in this tutorial are found here on Kenai:

http://kenai.com/projects/org-simplejava

2) Establish your new project, generate MIME type recognizer and generate data loader.

Step 1: Create a new NetBeans module.

Choose New Project | NetBeans Modules | Module. Type "Simplejava" as the project name, choose an appropriate folder as the Project Location, check "Standalone Module", and click Next.

Type "org.simplejava" as the Code Name Base, "Simple Java" as the Module Display Name and check "Generate XML Layer".

The IDE will create the module on disk and display its project structure in the IDE.


Step 2: Generate DataObject and MIME type resolver for your language.

Select "Simple Java" project node, New > Other... > Module Development > File Type. Select "text/x-simplejava" as a MIME Type, "sj" as a Extension and "SJ" as a Class Name Prefix. IDE will generate "SJDataObject.java", "SJResolver.xml", "SJTemplate.sj" files, and add some registrations to your "layer.xml" file.

If you would like to know more about stuff generated by IDE to your module now, follow these links:

You can Compile your project and Run NetBeans with this new module (select "Simple Java" project root node and select "Run" from pop-up menu). NetBeans is able to recognize "sj" files and open them in text editor. Now we will focus on a basic syntax coloring.

3) Syntax Coloring.

Syntax Coloring is a basic feature of every programming language support. In order to implement Syntax Coloring you need some Lexer (Tokenizer, Lexical Analyser). See http://en.wikipedia.org/wiki/Lexical_analyser for more info about lexical analyses. There are several ways how to create Lexer for your language. You can implement lexer by yourself, or use some existing tool that generates lexer implementation from some formal descriprion of your language (JavaCC, ANTLR, Flex...). If you would like to implement your lexer directly, you should implement "org.netbeans.spi.lexer.LanguageHierarchy", "org.netbeans.api.lexer.TokenId" and "org.netbeans.spi.lexer.Lexer" classes form NetBeans "lexer" module. See "http://bits.netbeans.org/dev/javadoc/org-netbeans-modules-lexer/overview-summary.html" for more info about Lexer APIs, and some real implementations: "java.lexer/src/org/netbeans/lib/java/lexer/JavaLexer.java", "languages.manifest/src/org/netbeans/modules/languages/manifest/MfLexer.java" or "cnd.lexer/src/org/netbeans/modules/cnd/lexer/ShLexer.java".

But we will use "JavaCC" to generate lexer in this tutorial. So download javacc-4.0.zip from "https://javacc.dev.java.net" and unpack it to some "/myjavacc40" folder. We will use grammar specified in "/myjavacc40/examples/JavaGrammars/1.5/Java1.5.jj" file. You can try to use some different version of JavaCC, but there can be some differences in generated files.

If you are using ANTLR to generate your lexer and parser you can read the ANTLR adaptation tutorial.

Step 3: Generate "Simple Java" lexer via JavaCC.

Create "org.simplejava.jcclexer" package in your "Simple Java" project, and copy "/myjavacc40/examples/JavaGrammars/1.5/Java1.5.jj" and "/myjavacc40/examples/JavaGrammars/1.5/Token.java" files there. Add "package org.simplejava.jcclexer;" line to "Java1.5.jj" file after "PARSER_BEGIN(JavaParser)" line:

PARSER_BEGIN(JavaParser)

package org.simplejava.jcclexer;

import java.io.*;
import java.util.*;

And add "package org.simplejava.jcclexer;" line to second line of "Token.java" file after "/* Generated By:JavaCC: Do not edit this line. Token.java Version 3.0 */":

/* Generated By:JavaCC: Do not edit this line. Token.java Version 3.0 */
package org.simplejava.jcclexer;

"Java1.5.jj" file contains descriptions of tokens for Java parser. Thats nearly what we need for our Java lexer. But there are some differences. Lexer defined for parser hides some types of tokens - typically comments and whitespaces. But we need to see such tokens in NetBeans lexer, because we need to define special colors for comments. So we need to change that in our JavaCC file. We should change:

SKIP :
{
  " "
| "\t"
| "\n"
| "\r"
| "\f"
}

to:

TOKEN :
{
  < WHITESPACE:
  " "
| "\t"
| "\n"
| "\r"
| "\f">
}

and change all SPECIAL_TOKEN definitions:

SPECIAL_TOKEN :
{
  <SINGLE_LINE_COMMENT: "//" (~["\n","\r"])* ("\n" | "\r" | "\r\n")?>
}

<IN_FORMAL_COMMENT>
SPECIAL_TOKEN :
{
  <FORMAL_COMMENT: "*/" > : DEFAULT
}

<IN_MULTI_LINE_COMMENT>
SPECIAL_TOKEN :
{
  <MULTI_LINE_COMMENT: "*/" > : DEFAULT
}

to TOKEN definitions:

TOKEN :
{
  <SINGLE_LINE_COMMENT: "//" (~["\n","\r"])* ("\n" | "\r" | "\r\n")?>
}

<IN_FORMAL_COMMENT>
TOKEN :
{
  <FORMAL_COMMENT: "*/" > : DEFAULT
}

<IN_MULTI_LINE_COMMENT>
TOKEN :
{
  <MULTI_LINE_COMMENT: "*/" > : DEFAULT
}

As we will use our "org.simplejava.jcclexer.Java1.5.jj" grammar file for tokenizer only, we can simplify it. Set BUILD_PARSER property to false:

options {
  JAVA_UNICODE_ESCAPE = true;
  ERROR_REPORTING = false;
  STATIC = false;
  JDK_VERSION = "1.5";
  BUILD_PARSER = false;
}

Delete JavaParser class body:

PARSER_BEGIN(JavaParser)

package org.simplejava.jcclexer;

public class JavaParser {}

PARSER_END(JavaParser)

And delete:

/*****************************************
 * THE JAVA LANGUAGE GRAMMAR STARTS HERE *
 :::***************************************/
and rest of "Java1.5.jj" file.

"Java1.5.jj" file is ready now, and we can "compile" it from command line now:

cd /myprojects/simplejava/src/org/simplejava/jcclexer
/myjavacc40/bin/javacc Java1.5.jj

The JavaCC parser generator will generate the "JavaCharStream.java", "JavaParserConstants.java", "JavaParserTokenManager.java", "ParseException.java", and "TokenMgrError.java" files into your "org.simplejava.jcclexer" package. All the files should be compilable:

Image:fig-1_How_to_create_support_for_a_new_language.png

We've now completed the JavaCC part of the tutorial. The time has come to use the generated files to create our NetBeans Lexer plugin.

Step 4: Create NetBeans Lexer plugin.

We should define dependency on Lexer module first: select "Simple Java" project root node, select "Properties" item form popup menu, check "Show Non-API Modules", and select "Libraries", "Add...". Now you should search for "Lexer" (in "Filter" input line), and add it to libraries.

Create "org.simplejava.lexer" package in your project now.

The first class we have to implement is TokenId. TokenId represents one type of token. It has three properties: name (unique name of this token type like "KEYWORD_IF"), id (unique number) and primaryCategory. primaryCategory can be used if you want to share the same tokens coloring color for more token types:

import org.netbeans.api.lexer.Language;
import org.netbeans.api.lexer.TokenId;

public class SJTokenId implements TokenId {

    private final String        name;
    private final String        primaryCategory;
    private final int           id;
    
    SJTokenId (
        String                  name,
        String                  primaryCategory,
        int                     id
    ) {
        this.name = name;
        this.primaryCategory = primaryCategory;
        this.id = id;
    }

    @Override
    public String primaryCategory () {
        return primaryCategory;
    }

    @Override
    public int ordinal () {
        return id;
    }

    @Override
    public String name () {
        return name;
    }
}

LanguageHierarchy contains list of token types for our language, and creates a new instances of our Lexer:

import java.util.*;
import org.netbeans.spi.lexer.*;

public class SJLanguageHierarchy extends LanguageHierarchy<SJTokenId> {

    private static List<SJTokenId>  tokens;
    private static Map<Integer,SJTokenId>
                                    idToToken;

    private static void init () {
        tokens = Arrays.<SJTokenId> asList (new SJTokenId[] {
            //[PENDING]
        });
        idToToken = new HashMap<Integer, SJTokenId> ();
        for (SJTokenId token : tokens)
            idToToken.put (token.ordinal (), token);
    }

    static synchronized SJTokenId getToken (int id) {
        if (idToToken == null)
            init ();
        return idToToken.get (id);
    }

    protected synchronized Collection<SJTokenId> createTokenIds () {
        if (tokens == null)
            init ();
        return tokens;
    }

    protected synchronized Lexer<SJTokenId> createLexer (LexerRestartInfo<SJTokenId> info) {
        return new SJLexer (info);
    }

    protected String mimeType () {
        return "text/x-simplejava";
    }
}

Lexer implementation reads input text and returns tokens for it. But our Lexer implementation delegates to lexer generated by JavaCC:

class SJLexer implements Lexer<SJTokenId> {

    private LexerRestartInfo<SJTokenId> info;
    private JavaParserTokenManager javaParserTokenManager;


    SJLexer (LexerRestartInfo<SJTokenId> info) {
        this.info = info;
        JavaCharStream stream = new JavaCharStream (info.input ());
        javaParserTokenManager = new JavaParserTokenManager (stream);
    }

    public org.netbeans.api.lexer.Token<SJTokenId> nextToken () {
        Token token = javaParserTokenManager.getNextToken ();
        if (info.input ().readLength () < 1) return null;
        return info.tokenFactory ().createToken (SJLanguageHierarchy.getToken (token.kind));
    }

    public Object state () {
        return null;
    }

    public void release () {
    }
}

We can register our plugin now. Add following code to your SJTokenId file:

    private static final Language<SJTokenId> language = new SJLanguageHierarchy ().language ();

    public static final Language<SJTokenId> getLanguage () {
        return language;
    }

and following registration to your layer.xml file:

    <folder name="Editors">
        <folder name="text">
            <folder name="x-simplejava">
                ... some current registrations ...
                <file name="org-netbeans-modules-editor-NbEditorKit.instance"/>
                <file name="language.instance">
                    <attr name="instanceCreate" methodvalue="org.simplejava.lexer.SJTokenId.getLanguage"/>
                    <attr name="instanceOf" stringvalue="org.netbeans.api.lexer.Language"/>
                </file>
            </folder>
        </folder>
    </folder>

See more information about Lexer APIs here.

Your source structure should now look as follows:

Image:fig-6_How_to_create_support_for_a_new_language.png

Both parts of our lexer job are done (NetBeans Lexer plugin and real implementation of lexer generated by JavaCC.) Now we need to connect them together.

Step 5: Connect your NetBeans Lexer plugin with lexer generated by JavaCC.

We should reimplement JavaCharStream first:

public class JavaCharStream {

    private LexerInput input;

    static boolean staticFlag;

    public JavaCharStream (LexerInput input) {
        this.input = input;
    }

    JavaCharStream (Reader stream, int i, int i0) {
        throw new UnsupportedOperationException ("Not yet implemented");
    }

    JavaCharStream (InputStream stream, String encoding, int i, int i0) throws UnsupportedEncodingException {
        throw new UnsupportedOperationException ("Not yet implemented");
    }

    char BeginToken () throws IOException {
        return readChar();
    }

    String GetImage () {
        return input.readText ().toString ();
    }
    
     public char[] GetSuffix (int len) {
        if (len > input.readLength ())
            throw new IllegalArgumentException ();
        return input.readText (input.readLength () - len, input.readLength ()).toString ().toCharArray ();
     }

    void ReInit (Reader stream, int i, int i0) {
        throw new UnsupportedOperationException ("Not yet implemented");
    }

    void ReInit (InputStream stream, String encoding, int i, int i0) throws UnsupportedEncodingException {
        throw new UnsupportedOperationException ("Not yet implemented");
    }

    void backup (int i) {
        input.backup (i);
    }

    int getBeginColumn () {
        return 0;
    }

    int getBeginLine () {
        return 0;
    }

    int getEndColumn () {
        return 0;
    }

    int getEndLine () {
        return 0;
    }

    char readChar () throws IOException {
        int result = input.read();
        if (result == LexerInput.EOF) {
            throw new IOException("LexerInput EOF");
        }
        return (char) result;
    }
}

This version of JavaCharStream reads input chars from LexerInput, in place of standard InputStream. Additionally the JavaParserTokenManager created by javacc is designed to work with a java.io.Reader and recognizes a <EOF> when the io.Reader throws and IOException. However from the LexerInput API ("org.netbeans.spi.lexer.LexerInput" "... It logically corresponds to java.io.Reader but its read() method does not throw any checked exception. ...", hence the BeginToken and the readChar methods validate the returned char and throw the exception if necesary..


And now we should introduce JavaCC token types to our Lexer plugin. Just rewrite token types defined in JavaParserConstants file generated by JavaCC:

public interface JavaParserConstants {
  int EOF = 0;
  int WHITESPACE = 1;
  int SINGLE_LINE_COMMENT = 4;
  int FORMAL_COMMENT = 5;
  int MULTI_LINE_COMMENT = 6;
  int ABSTRACT = 8;
  int ASSERT = 9;
  int BOOLEAN = 10;
  int BREAK = 11;
  int BYTE = 12;
  int CASE = 13;
  int CATCH = 14;
  int CHAR = 15;
  ...

to your SJLanguageHierarchy file:

import static org.simplejava.jcclexer.JavaParserConstants.*;

public class SJLanguageHierarchy extends LanguageHierarchy<SJTokenId> {

    private static List<SJTokenId>  tokens;
    private static Map<Integer,SJTokenId>
                                    idToToken;

    private static void init () {
        tokens = Arrays.<SJTokenId> asList (new SJTokenId[] {
            new SJTokenId ("EOF", "whitespace", EOF),
            new SJTokenId ("WHITESPACE", "whitespace", WHITESPACE),
            new SJTokenId ("SINGLE_LINE_COMMENT", "comment", SINGLE_LINE_COMMENT),
            new SJTokenId ("FORMAL_COMMENT", "comment", FORMAL_COMMENT),
            new SJTokenId ("MULTI_LINE_COMMENT", "comment", MULTI_LINE_COMMENT),
            new SJTokenId ("ABSTRACT", "keyword", ABSTRACT),
            new SJTokenId ("ASSERT", "keyword", ASSERT),
            new SJTokenId ("BOOLEAN", "keyword", BOOLEAN),
            new SJTokenId ("BREAK", "keyword", BREAK),
            new SJTokenId ("BYTE", "keyword", BYTE),
            new SJTokenId ("CASE", "keyword", CASE),
            new SJTokenId ("CATCH", "keyword", CATCH),
            new SJTokenId ("CHAR", "keyword", CHAR),
            ...

Token category should correspond to the color you want to see in the editor. Notice that we MUST keep token identifiers (numbers)!!!

Go here to access the completed file:

http://kenai.com/projects/org-simplejava

We have JavaCC generated lexer integrated to IDE. And we can define concrete colors for our tokens now.

Step 6: Define concrete colors for your token types.

Color for token types are defined declaratively in one xml file. Create FontAndColors.xml file in /myprojects/simplejava/src/org/simplejava folder:

<!DOCTYPE fontscolors PUBLIC 
    "-//NetBeans//DTD Editor Fonts and Colors settings 1.1//EN"  
    "http://www.netbeans.org/dtds/EditorFontsColors-1_1.dtd">
<fontscolors>
    <fontcolor name="character" default="char"/>
    <fontcolor name="errors" default="error"/>
    <fontcolor name="identifier" default="identifier"/>
    <fontcolor name="keyword" default="keyword" foreColor="red"/>
    <fontcolor name="literal" default="keyword" />
    <fontcolor name="comment" default="comment"/>
    <fontcolor name="number" default="number"/>
    <fontcolor name="operator" default="operator"/>
    <fontcolor name="string" default="string"/>
    <fontcolor name="separator" default="separator"/>
    <fontcolor name="whitespace" default="whitespace"/>
    <fontcolor name="method-declaration" default="method">
        <font style="bold" />
    </fontcolor>
</fontscolors>

This file defines how to visualize tokens produced by lexer.

"fontcolor" tag properties:

  • name: Name or primaryCategory of your token (or tokens).
  • default: Name of default coloring. All properies that are not defined explicitly are inherited from this default coloring. Default coloring is customizable in

NetBeans main menu > Tools > Options > Fonts & Colors, if you select "Language: All Languages".

  • foreColor: Foreground color.
  • bgColor: Background color.
  • underline: Underlined color. Token will be underlined if specified.
  • strikeThrough: Strike through color.
  • waveUnderlined: Wave underlined color.

"fontcolor" tag can contain nested font tag. "font" tag has following properties:

  • name: Name of font.
  • size: Font size.
  • style: Bold or italic style.

See DTD for this file here.

List of all token colors are editable in NetBeans Options Dialog. We should provide some short example text written in our language. So create file "/myprojects/simplejava/src/org/simplejava/SimpleJavaExample.sj" with following content:

/**
 * SimpleJavadoc comment for <code>SimpleJavaExample</code> class.
 * @author Simple Joe Smith
 */
public class SimpleJavaExample {
    
    @Deprecated public String method (int param) {
        return "SimpleString " + '-' + 1.2;
    }// line comment
}


Do not forget to register your "FontAndColors.xml" and "SimpleJavaExample.sj" files in your "layer.xml":

    <folder name="Editors">
        <folder name="text">
            <folder name="x-simplejava">
                ... some current registrations ...
                <attr name="SystemFileSystem.localizingBundle" stringvalue="org.simplejava.Bundle"/>
                <folder name="FontsColors">
                    <folder name="NetBeans">
                        <folder name="Defaults">
                            <file name="FontAndColors.xml" url="FontAndColors.xml">
                                <attr name="SystemFileSystem.localizingBundle" stringvalue="org.simplejava.Bundle"/>
                            </file>
                        </folder>
                    </folder>
                </folder>
            </folder>
        </folder>
    </folder>
    <folder name="OptionsDialog">
        <folder name="PreviewExamples">
            <folder name="text">
                <file name="x-simplejava" url="SimpleJavaExample.sj"/>
            </folder>
        </folder>
    </folder>

Your "/myprojects/simplejava/src/org/Bundle.properties" file should contain localized names of your language, and token types:

text/x-simplejava=Simple Java
character=Character
errors=Error
identifier=Identifier
keyword=Keyword
literal=Literal
comment=Comment
number=Number
operator=Operator
string=String
separator=Separator
whitespace=Whitespace
method-declaration=Method Declaration

Syntax coloring is finished! You can rename some Foo.java file to Foo.sj and open it in the NetBeans editor. It should be colored, all keywords should be red:

Image:fig-2_How_to_create_support_for_a_new_language.png

Go to Tools > Options > Fonts & Colors. The Language drop-down list should contain a "Simple Java" entry, as you can see below. If you select it you will see a list of your token types and your "Simple Java" example text. You can change the colors for your tokens here too:

Image:fig-3_How_to_create_support_for_a_new_language.png

4) Parser integration

Step 7: Generate "Simple Java" parser by JavaCC.

Create "org.simplejava.jccparser" package and copy "/myjavacc40/examples/JavaGrammars/1.5/Java1.5.jj" and "/myjavacc40/examples/JavaGrammars/1.5/Token.java" files there. Add "package org.simplejava.jcclexer;" line to "Java1.5.jj" file after "PARSER_BEGIN(JavaParser)" line. And add "package org.simplejava.jcclexer;" line to second line of "Token.java" file after "/* Generated By:JavaCC: Do not edit this line. Token.java Version 3.0 */".

Compile "Java1.5.jj" file.

We have "Simple Java" parser now, and we can integrate it to NetBeans now.

Step 8: Integrate parser to NetBeans.

Implementation of Parser API is similair to other NetBeans APIs. You have to implement Parser class, ParserFactory and register ParserFactory in your layer.xml file.

So, create "org.simplejava.parser" package.

Add dependency on "Parsing API" (select root node of your module > Properties > Libraries > Add...).

Create "SJParserFactory" class in "parser" package:

public class SJParserFactory extends ParserFactory {

    @Override
    public Parser createParser (Collection<Snapshot> snapshots) {
        return new SJParser ();
    }
}

Create "SJParser" class:

public class SJParser extends Parser {

    private Snapshot snapshot;
    private JavaParser javaParser;

    @Override
    public void parse (Snapshot snapshot, Task task, SourceModificationEvent event) {
        this.snapshot = snapshot;
        Reader reader = new StringReader (snapshot.getText ().toString ());
        javaParser = new JavaParser (reader);
        try {
            javaParser.CompilationUnit ();
        } catch (org.simplejava.jccparser.ParseException ex) {
            Logger.getLogger (SJParser.class.getName()).log (Level.WARNING, null, ex);
        }
    }

    @Override
    public Result getResult (Task task) {
        return new SJParserResult (snapshot, javaParser);
    }

    @Override
    public void cancel () {
    }

    @Override
    public void addChangeListener (ChangeListener changeListener) {
    }

    @Override
    public void removeChangeListener (ChangeListener changeListener) {
    }

    
    public static class SJParserResult extends Result {

        private JavaParser javaParser;
        private boolean valid = true;

        SJParserResult (Snapshot snapshot, JavaParser javaParser) {
            super (snapshot);
            this.javaParser = javaParser;
        }

        public JavaParser getJavaParser () throws org.netbeans.modules.parsing.spi.ParseException {
            if (!valid) throw new org.netbeans.modules.parsing.spi.ParseException ();
            return javaParser;
        }

        @Override
        protected void invalidate () {
            valid = false;
        }
    }
}

And register ParserFactory in your layer:

    <folder name="Editors">
        <folder name="text">
            <folder name="x-simplejava">
                ... some current registrations ...
                <file name="org-simplejava-parser-SJParserFactory.instance"/>
            </folder>
        </folder>
    </folder>

Your parser generated by JavaCC is registerred in NetBeans now. You can compile and run "Simple Java" module. But your parser will never be called. There is simple reason. Nobody asks for parser results, there is no client of "Simple Java" parser. So, we should create some client for our parser.

Step 9: Adding some basic error recovery to our parser.

We will create a first client of SJParser now. This client (task) will show syntax errors in editor gutter. But we should do several modifications to our parser first. Our parser throws ParseException when it finds first error in source code. This is default behaviour of parsers genearted by JavaCC. But in the IDE we need to detect more than one syntax error. So we should try to add some simple error recovery to our parser:

1) Change "ERROR_REPORTING = false;" to "ERROR_REPORTING = true;".

2) Add "import java.util.*;" to your Java1.5.jj file.

3) Add following method to your JavaParser body:

   public List<ParseException> syntaxErrors = new ArrayList<ParseException> ();

   void recover (ParseException ex, int recoveryPoint) {
      syntaxErrors.add (ex);
      Token t;
      do {
          t = getNextToken ();
      } while (t.kind != EOF && t.kind != recoveryPoint);
   }

4) Catch ParseExceptions in FieldDeclaration, MethodDeclaration and Statement :

void CompilationUnit():
{}
{
    try {
        [LOOKAHEAD((Annotation())*"package")PackageDeclaration() ]
        ( ImportDeclaration() )*
        ( TypeDeclaration() )*
        ( < "\u001a" > )?
        ( <STUFF_TO_IGNORE: ~[]> )?
        <EOF>
    } catch (ParseException ex) {
        recover (ex, SEMICOLON);
    }
}
...
void FieldDeclaration(int modifiers):
{}
{
    try {
      // Modifiers are already matched in the caller
      Type() VariableDeclarator() ( "," VariableDeclarator() )* ";"
    } catch (ParseException ex) {
        recover (ex, SEMICOLON);
    }
}
...
void MethodDeclaration(int modifiers):
{}
{
    try {
        // Modifiers already matched in the caller!
        [TypeParameters() ]
        ResultType()
        MethodDeclarator() ["throws"NameList() ]
        ( Block() | ";" )
    } catch (ParseException ex) {
        recover (ex, SEMICOLON);
    }

}
...
void Statement():
{}
{
    try {
          LOOKAHEAD(2)
          LabeledStatement()
        |
          AssertStatement()
        |
          Block()
        |
          EmptyStatement()
        |
          StatementExpression() ";"
        |
          SwitchStatement()
        |
          IfStatement()
        |
          WhileStatement()
        |
          DoStatement()
        |
          ForStatement()
        |
          BreakStatement()
        |
          ContinueStatement()
        |
          ReturnStatement()
        |
          ThrowStatement()
        |
          SynchronizedStatement()
        |
          TryStatement()
    } catch (ParseException ex) {
        recover (ex, SEMICOLON);
    }
}

Recompile Java1.5.jj now:

cd /myprojects/simplejava/src/org/simplejava/jcclexer
/myjavacc40/bin/javacc Java1.5.jj

We have added some very basic error recovery to our parser. So we can display syntax errors in editor now.

Step 10: Syntax errors reporting.

Now we are ready to implement our first ParserResultTask. This task consists from three usual steps: create some factory (TaskFactory), create task (ParserResultTask) and register factory in layer.xml file:

Create "SyntaxErrorsHighlightingTaskFactory" class in "org.simplejava.parser" package:

import org.netbeans.modules.parsing.api.Snapshot;
import org.netbeans.modules.parsing.spi.SchedulerTask;
import org.netbeans.modules.parsing.spi.TaskFactory;

public class SyntaxErrorsHighlightingTaskFactory extends TaskFactory {

    @Override
    public Collection<? extends SchedulerTask> create (Snapshot snapshot) {
        return Collections.singleton (new SyntaxErrorsHighlightingTask ());
    }
}

Create "SyntaxErrorsHighlightingTask" class:

class SyntaxErrorsHighlightingTask extends ParserResultTask {

    public SyntaxErrorsHighlightingTask () {
    }

    @Override
    public void run (Result result, SchedulerEvent event) {
        try {
            SJParserResult sjResult = (SJParserResult) result;
            List<ParseException> syntaxErrors = sjResult.getJavaParser ().syntaxErrors;
            Document document = result.getSnapshot ().getSource ().getDocument (false);
            List<ErrorDescription> errors = new ArrayList<ErrorDescription> ();
            for (ParseException syntaxError : syntaxErrors) {
                Token token = syntaxError.currentToken;
                int start = NbDocument.findLineOffset ((StyledDocument) document, token.beginLine - 1) + token.beginColumn - 1;
                int end = NbDocument.findLineOffset ((StyledDocument) document, token.endLine - 1) + token.endColumn;
                ErrorDescription errorDescription = ErrorDescriptionFactory.createErrorDescription (
                    Severity.ERROR,
                    syntaxError.getMessage (),
                    document,
                    document.createPosition (start),
                    document.createPosition (end)
                );
                errors.add (errorDescription);
            }
            HintsController.setErrors (document, "simple-java", errors);
        } catch (BadLocationException ex1) {
            Exceptions.printStackTrace (ex1);
        } catch (org.netbeans.modules.parsing.spi.ParseException ex1) {
            Exceptions.printStackTrace (ex1);
        }
    }

    @Override
    public int getPriority () {
        return 100;
    }

    @Override
    public Class<? extends Scheduler> getSchedulerClass () {
        return Scheduler.EDITOR_SENSITIVE_TASK_SCHEDULER;
    }

    @Override
    public void cancel () {
    }
}

And register TaskFactory in your layer:

    <folder name="Editors">
        <folder name="text">
            <folder name="x-simplejava">
                ... some current registrations ...
                <file name="org-simplejava-parser-SyntaxErrorsHighlightingTaskFactory.instance"/>
            </folder>
        </folder>
    </folder>

Check that your source structure is now as follows:

Image:fig-5_How_to_create_support_for_a_new_language.png

When you install the module again, you should see syntax errors marked as follows:

Image:fig-4_How_to_create_support_for_a_new_language.png

Our simplejava parser is now integrated into the NetBeans Platform, and we can continue coding new features based on it, as done above for syntax error checking.

Apendix A: List of APIs referenced from this tutorial

Name of API Description Name of module
File Systems and MIME Type Resolvers Manipulate files and their MIME types. openide.filesystems
Data Systems Define icon, actions, and more for your files. openide.loaders
Lexer API Define tokens of your language and way how to visualise them. lexer
Parsing API Integrate a parser with the rest of the IDE. parsing.api
Not logged in. Log in, Register

By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo