Ruby Code Overview

(This document is incomplete)

This document attempts to describe the various components involved in the NetBeans Ruby IDE. It is aimed primarily for developers wanting to join the project and getting oriented.

The Code

The code lives in the Mercurial modules that contain "ruby" in their names. This also corresponds to a netbeans.org project named "ruby" which has its own bugtracking system category, its own mailing lists (for development discussion, dev@ruby.netbeans.org, for users, users@ruby.netbeans.org, for code commits, cvs@ruby.netbeans.org, and for bug/issue traffic, issues@ruby.netbeans.org).

The NetBeans modules

After checking out the Mercurial repository, look for modules that begin with "ruby" (these all correspond to NetBeans modules as well).

• ruby/build.xml - An ant file used to build all the Ruby modules. Not used by production builds. Also contains targets to run all the unit tests.
• ruby.editing/ - Most of the editing support for Ruby: syntax highlighting, code completion, rdoc handling, parsing, reformatting, etc.
• o.jruby.distro/ - Contains the JRuby source code (as a .zip file), which it patches and then builds into a NetBeans module. This module is used by the IDE to parse Ruby code such that it can analyze it (for code completion, refactoring, etc)
• ruby.debug-commons-java/ - A wrapper module for the Source Forge project of the same name which is used to talk to Ruby debugger backends
• ruby.debugger/ - The debugger UI and logic for Ruby
• gsf/ - IDE infrastructure intended to be sharable by many dynamic languages; there should be nothing Ruby specific there
• ruby.help/ - A JavaHelp module providing help for the Ruby support
• ruby.javaint/ - Java integration module which lets Ruby projects which are run under JRuby to access jar files. Will eventually support code completion into Java classes etc.
• o.jruby/ - Similar to the bundledjruby module, this builds JRuby, but this time using JRuby's own build files to build a complete JRuby installation. This is (optionally) bundled with the IDE. It also bundles Rails gems.
• ruby.hints/ - Contains code for quickfixes and hints
• ruby.extrahints - Additional experimental hints (not part of the base build; available on Auto Update)
• ruby.kit/ - A simple "wrapper" module which just adds dependencies on everything else. This was necessary under the old Auto Update client to make it easy to install everything Ruby related. We may no longer need it with the new plugin manager in NetBeans 6.
• o.kxml2/ - Wrapper for a library used by the debugger.
• ruby.platform/ - Module which manages Ruby platforms/interpreters. Should eventually be similar to the Java platform manager which makes it easy to install and manage multiple JDKs; for now it just manages a single Ruby interpreter controlled by setttings in the Options panel.
• ruby.projects/ - A project-type implementation for plain Ruby projects. (Is not used by Rails projects.)
• ruby.railsprojects/ - A project-type implementation for Ruby-On-Rails projects.
• ruby.rake/ - Some support for "rake-based" projects, although in truth the metadata is still stored in ant-like properties syntax in the nbproject directory.
• ruby.refactoring/ - Refactoring implementations of Find Usages and Rename - and hopefully more refactorings in the future. RHTML refactoring is also handled here.
• ruby.rhtml/ - Support for RHTML/ERB files.

Editing

The ruby.editing module contains the core Ruby support.

The RubyMimeTypeResolver recognizes files as Ruby files by analyzing both the extension as well as the file header for some files. This is for example where .mab files are known to be Ruby files.

Lexing

The RubyLanguage and RubyTokenId classes define basic language and token attributes for Ruby.

The RubyLexer lexes Ruby sources into lexical token that are used for a wide variety of purposes. Syntax highlighting is one obvious use, but features like highlighting matching parentheses and do/end tags, automatically inserting "end" when pressing return on a line that needs it, etc. all rely on the syntax tokens.

The RubyCommentLexer lexes Ruby comments into logical tokens like "link", "html tag", "rdoc directive", and obviously "normal comment".

The RubyStringLexer lexes Ruby strings into logical tokens like "normal string" and "escaped string".

The *TokenId classes define embedding scenarios such that a String can contain Ruby code, Ruby code can contain strings and comments, etc. Arbitrarily deep nesting is allowed.

Parsing

The RubyParser uses JRuby to produce an AST (abstract syntax tree) from a source. It is invoked by the IDE as necessary (e.g. when the source has been modified, or when it's first opened, etc.) and the parse tree is handed to many other operations, such as the CodeCompleter.

Lexing and Parsing

Most features rely both on lexing information as well as parsing information.

The CodeCompleter is invoked to produce completion results for a buffer, given an offset and a parse tree. It often also uses the syntax tokens to determine what to do. For example, it might first look at the token hierarchy under the caret and see if it's inside a regular expression, and if so, offer regexp matches instead of Ruby matches.

The BracketCompleter is perhaps poorly named; it is used to for example automatically insert a closing bracket, ], after you type an opening bracket, . And if you type the closing bracket yourself, it will figure out that it should just type through the existing closing bracket. However, it's used for a wide variety of other purposes; handling single quotes, double quotes and parentheses, automatically inserting "end" when missing", automatically reindenting lines when you for example type "end", and so on. It's really a keypress handler. The {{StructureAnalyzer}} is used to walk over an AST and extract some critical "structure" information from it. This structure is used for various purposes, such as populating the Navigator/Outline view. The {{RDocFormatter}} is used to take rdoc-formatted comments and produce HTML from them. This is used inside code completion dialogs for example. TODO - describe astOffset versus lexOffset