Part21Schliemann

Support Missing Languages using Schliemann

Carsten Zerbst

As every programmer knows the special file format or programming language you are working with ist not supported by your favourite IDE. This was true for VIM or xemacs and is still true in the days of Netbeans. And in most cases you neither need a full blown support like for Java nor have the time to implement it. That is where Schliemann comes into the game, a framwork within Netbeans to support you creating basic support for you favourite language.

The Generic Languages Framework (codename Schliemann) offers an easy way to support you language with the Netbeans IDE. This tutorial shows how to support the common STEP Physical File (ISO 10303 Part21) format, a file format pretty much used in the exchange off CAD and Produkt Structure Exchange. The complete STEP thing offered complete MDA support for already 10 years, including an own language to define models and restrictions (Part11) and the Part 21 defining how to save them to a file. The Part 21 semantics is pretty rich and shows a lot of features needed in other file formats or languages as well.

In the end one has a modele supporting syntax highlighting, shows the internal structure of the file in the Navigator and is able to find STEP entities referencing a certain line and go from a reference to the referenced line.

Image:part21module_Part21Schliemann.png


Create a Module

To support a new language using Schliemann one needs a new Netbenas Module for a start.

Image:module_Part21Schliemann.png


This is plain vanilla Netbeans module, as next step one need to add Language Support using the New on the project node.

Image:languageSupport_Part21Schliemann.png

Normally step files should have the MIME type application/x-step, but Netbeans seems to have difficulties with MIME types not starting with text. The files are recognized by their extension, so common file extensions are given as space separated list.

Image:mimeType_Part21Schliemann.png


Tokens

The wizard creates a number off files for the language support with language.nbs as most important. It will later on contain the description of your language, for a start it is file with some dummy syntax. There is some information on this Wiki SchliemannNBSLanguageDescription on the NBS syntax, but looking at the real examples like the JavaScript support based on Schliemann it is not complete. Perhaps one should start page containing syntax descriptions as e.g. JavaCC has.

The first thing to do to support you language is describe the atomic units or tokens used. Each token description has the format TOKEN:$name:$description. The description is captured in normal parents "()", it then contains all characters possible in the token. The token description below contains different descriptions:

  • Simple Tokens containing known keywords,
TOKEN:ISO:("ISO-10303-21") will exactly mactch ISO-10303-21
  • Tokens containing one ore more characters from a list

TOKEN:whitespace:( " " "\t" "\n" "\r" + ) will match every sequnce of one or more blanks, tabs, newline and linefeeds

  • Tokens containing every character from a start to a stop sequence

TOKEN:block_comment: ("/*" - "*/) will match any number off characters starting with /* and ending with */ including newlines

  • Token description containing all but (^) certain characters

TOKEN:line_comment:("//" ^"\n""\r"*) will match any number off characters staring with // followd by everything except newline and linefeed

  • Token description containing ranges off characters,

TOKEN:enumeration:( "." "A"-"Z" "A"-"Z" "0"-"9"* ".") will match everything starting with a dot, is followed by one character from the A-Z range, may contain zero or more chacters from the A-Z + 0-9 ranges and stops with a dot. Example .BEHIND.

  • Using +, ? and * one could describe how often a symbol occurs
  • The plus character defines a one to many counter on the described symbol
  • The asterisk defines a none to many counter on the described symbol
  • The question mark defines a zero to one counter on the described symbol

TOKEN:real:( "+" "-"? "0"-"9"+ "." "0"-"9"* ) will match sequences which (may start with plus or minus) (start with one or more digits) (contain a dot) (may have trailing digits)


In most cases there a things like comments ore whitespace which should be ignored, this could be done using the SKIP statement.

Image:tokens_Part21Schliemann.png


Syntax

When you are finished defining the tokens you need to define the syntax or grammar, so how your file format is made from the tokens. A syntax rule has the format $SyntaxName = $SyntaxDescription ;.


  • The toplevel syntax describing the entire file must be name S
  • Tokens must be referenced by their name in angled brackets, <ISO>
  • Syntax rules must end with an semicolon
  • Use the pipe to denote A or B, <star> | <dollar>
  • Use parens , plus and asterisk with the same meaning as within the token description

The grammar below defines the Part21 files.

  • Each file needs to start with the keyword ISO-10303-21 (token ISO) and end with END-ISO10303-21 (token ENDISO),

it should contain a header and data section multipleDataSection

  • The header section should contain at least three header entities, so I use +
  • The most basic block of header and data section is the ParamterList containing one to many parameters in braces
  • The paramter list is usually prefixed by a keyword describing the class
  • As there need to be references in the data section every entry has its own id (e.g. #23). This is not needed in the header section
  • To make it a little bit more complex there are typed parameters , e.g. PLANE_ANGLE_MEASURE in this example __#68=PLANE_ANGLE_MEASURE_WITH_UNIT(PLANE_ANGLE_MEASURE(0.01745329251994),#69);
  • To make it even a bit more complex multiple inheritance use complex type in Part21, e.g. #65=(GEOMETRIC_REPRESENTATION_CONTEXT(3) GLOBAL_UNIT_ASSIGNED_CONTEXT(( #66,#67,#71) ) REPRESENTATION_CONTEXT(,'external'));


Image:grammar_Part21Schliemann.png


As last step you need to define the right visual representation for you grammer:

Image:color_Part21Schliemann.png


When you think you are done with you file format, select the projectnode and click on "Install/Reload in Target Plattform". When you then open a file with the right extension you should see some kind off syntax highlighting. Normally I don't do due to some mistakes some where in the NBS file, but where ?

Token and AST View

There are some case where Schliemann loudly complains about errors, for example when you reference an non existing (misspelled) syntax. But in other cases just nothing happens. To aid you in those cases Netbeans contains a deeply hidden window showing the found tokens and the abstract syntax tree (AST). This is only available by a right mouselclick on the nbs node in the IDE where the module is loaded, Then select a file which should be highlighted and the tokens views (left, upperside), and AST ( left, lowerside) is displayed. Image:tokenASTViews_Part21Schliemann.png Image:tokenASTView_Part21Schliemann.png



Navigator

When creating something new like syntax highlighting I don't like to reinvent the wheel. Before starting you own syntax perhaps you should have a look at other, already existing nbs files. One example of an nbs file containing nearly everything is the http://hg.netbeans.org/main/file/c2a924c75049/languages.javascript/src/org/netbeans/modules/languages/javascript/|JavaScript.nbs as found in Netbeans Mercural server. In this file you see that it is possible to use the syntax as input for the Navigator. You could then klick on a certain entity in the Navigator and the editor will show the selected instance like with methods in Java. As you could see below in most cases it is pretty trivial to add Navigator support, one needs only to define a Navigator line for the desired syntax element. This element could have node name, a node icon and eventually a tooltip. A SimpleEntityInstance is representedt by item id ( == entity instance name ) and the class (== keyword) as node label, the tooltip contains the complete entity. The icons are reused from the language support module to have a homogeneous look with other navigators.


Image:navigator_Part21Schliemann.png

ComplexEntityInstances (which represent more or less multiple inhertitance classes) should have a list of their classes as node label, something not possible with the common NBS syntax. Fortunately its possible to a call static Java methods from within the NBS description, de.groygroy.part21module.Part21.navigatorComplexEntityInstance in this case. The method is called giving the SyntaxContext containing both the complete Document and the current AST Path. This path is current position == the AST node for the ComplexEntityInstance. Starting from that node first the ASTItem containing the id ( entity instance name) is retrieved and then all further nodes starting a subtype ( == SimpleRecord ) are searched and their value is used.


Image:navigatorJava_Part21Schliemann.png


The API part off Schliemann is more or less undocumented. Expect to got one off the following types:

  • ASTItem, one node in the AST tree. Abstract super class. Contains offset within document, length, and children List<ASTItem>
  • ASTNode, subtype of ASTItem, represents a syntax node. Contains syntax name (nt)
  • ASTToken, subtype of ASTItem, represents a token. Contains text in identifier variable


Actions

The next missing thing for a good Part21 support is off course a functionality to follow references within the file. Actions have to be defined in Java, the NBS file just references a method to check whether the action is currently enabled and the method (performer) itself.

Image:actions_Part21Schliemann.png


This first method should reposition the cursor from an item reference to the item itself. There have to be two methods in the class public static boolean enabledFindReferencing( ASTNode node, JTextComponent comp ) { which checks the the current position is on an entity reference and public static void gotoReference( final ASTNode rootNode, final JTextComponent comp ) { which searchs through the document for the item and sets the selection. As the methods get the document and now position and length from the ASTItems the selection is pretty easy to set once the desired ASTItem was found using:

__comp.select( target.getOffset(), target.getOffset() + target.getLength() );

           comp.setCaretPosition( target.getOffset() );

__

See the attached  Part21.java file containing the complete implementation.

The other way to navigate within an Part21 file is off course to find all entites referencing one particular entity. This findReferencing action could be seen in the first screenshot. For a real Netbeans like implementation it should have used the Refactoring API but the documentation is pretty thin. Therefore the module contains a simple Toplevel containing the found references. Get the complete project from Part21Module_Part21Schliemann.tgz.


Hidden Features

There are still some more features in the NBS files not explained in this document. MARK, COMPLETION, SEMANTIC_DECLARATION, SEMANTIC_CONTEXT, SEMANTIC_USAGE, PROPERTIES, COMMENT_LINE, BUNDLE






multipleDataSection

yes, technically there may be many data sections but I've never seen it in real life


Not logged in. Log in, Register

By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo