CndNb65RemoteDevelopmentDesignSpec

Design Specification for Remote Development.

Introduction

The following areas are covered by this document:

  • transport between local client and remote server
  • infrastructure for remote server setup
  • host/authorization info
  • tool chains for remote host
  • host status/monitor
  • infrastructure for running project actions on remote server
  • preparing project for remote running (Makefiles, configuration files)
  • executing tasks on remote server
  • synchronizing input/output (and different types of terminals)
  • debug routines (not covered by this spec)
  • sources and code model support
  • remote server includes handling
  • path mapping (not covered by this spec)
  • miscellaneous items

Overview

This is a brief check list of concepts described below:

  • For connections from a client to a remote server, we use the SSH2 protocol from OpenSSH (the Java implementation JSch)
  • New UI components will be provided to handle and store hosts/user properties
  • Project system and code model should have minimal awareness regarding local vs. remote operation
  • To provide remote system headers for code model, system headers will be copied to client-side cache
  • Source files are not replicated, user should setup file sharing to have them accessible from both client and server
  • Differentiation in file names on client and server will be resolved by special path mapping UI and engine
  • We can have a server-side presence (in the form of shell and possibly perl scripts) for all functionality which can't be covered by the above mechanisms

Transport

Transport provider: SSH

SSH provides for us the following functionality:

  • Remote Execution:
  • The ability to run user tasks and work with I/O streams
  • The ability to execute service commands on remote servers, e.g.:
  • Search for files in $PATH
  • Recognize remote Operating System
  • Validate file/directory existence
  • File Transfer Engine:
  • Using the ssh protocols, we can seamlessly copy files between client and server using Scp or Sftp protocols
  • Secure connection
  • SSH Server implementations are included in most Linux/Solaris distributions

Protocol provider: JSch -- Java Secure Channel (http://www.jcraft.com/jsch/)

This is a Java implementation of th SSH2 protocol and most of it's additional features. Pros:

  • It’s already included in NetBeans
  • Its open source
  • It’s under actively development
  • It's quite mature, and used by several serious applications such as Ant and Eclipse.

Toolchains and Servers

There are two approaches, from the user point of view, on working with connection between servers and toolchains. In the prototype we assumed the server is just a parameter of the toolchain, so changing toolchains in project properties changes the server as well. Another approach is to have a drop-down menu allowing the user to select a server. The toolchains displayed in the Tool Chain list would reflect the toolchains found on the selected server. This issue has not finalized and will be reviewed by HIE in the next few days.

Server Monitor

The UI has not yet been approved by HIE. It could be an additional window or a node in the Services tab.

Each server can be in one of the several states:

  • Offline/Not responding – which means we cannot work with this server
  • Alive – which means the server has been pinged successfully and can be used as a remote development server
  • Initializing – temporaty state while toolchain, system headers are being loaded

Infrastructure for Running Project Actions on the Remote Server

There are quite a few places in code where next two operations are required:

  • changing system calls to HostFacade calls
  • propagating HostFacade reference or remote host data to classes which should be aware of them.
  • applying file mappings

There is no good to enumerate them all. Only most complex topics are described.

Preparing project for remote running (Makefiles, configuration files)

Makefile generation requires special attention due to need in both remote system context and extensive use of path mappings.

Project and files properties also should be carefully checked to don't mess with mapped file names and client vs remote systems.

Executing Tasks on the Remote Server (Build, Run)

Executing Build and Run routines requires more tight integration with SSH, e.g., due to work with input and output streams. To handle this new interface Execution was extracted from cnd.execution.NativeExecution and new implementation for it RemoteExecution.

public interface Execution {
    int executeCommand(
            File runDirFile,
            String executable,
            String arguments,
            String[] envp,
            PrintWriter out,
            Reader in) throws IOException, InterruptedException;
    
    public void stop();
}

Note: I may make this into an abstract class which provides a factory method providing an implementation. The implementation would be either an instance of NativeExecution or RemoteExecution.

Console Types

  • Output Window: works for remote mode as well. But it's not appropriate for running user applications which requires input.
  • External Terminal: This requires some additional investigation. There are several approaches:
  • We can pass I/O between client and server and use one of the client's external terminal types
  • We can run a server terminal program and redirect it and display I/O in that terminal.

    Cons:
  • Requires $DISPLAY be set on the client and passed in the environment to the server
  • If the user wants to display on the terminal on the client, they must run an xhosted X server on the client and set $DISPLAY appropriately


Debugger

Suggested model: The gdb process will run on the server over a jsch connect. I/O (both user and debugger module) will be sent through the ssh connection.

Details TBD

Sources and Code Model Support

For correct code model behavior in remote mode, the following statements have to be fulfilled:

  • System headers for the remote server must be available in the client cache
  • Compiler settings should be correctly set for the remote server

Remote System Headers

Remote system headers will be cached on the client for Code Model needs.

The responsible class is cnd.remote.SystemIncludesUtils. It provides utility functions to copy system headers using scp (of sftp) over an ssh connection.

Loading system headers is a part of setting a remote toolchain.

Path Mapping

The agreed model is: There is only one instance of a source file which is accessible from both client and server. It can be:

  • Stored on a common file server, with access on both the client and development servers
  • In a shared folder on the server (via NFS, Samba or something equivalent)
  • In a shared folder on the client

Some cases will require path mapping. If the Windows client has mounted an NFS partition server1:/work/projects on the Z: drive, then the path might be Z:/NetBeansProjects/Args_1/src/args.c on Windows, but /work/projects/NetBeansProjects/Args_1/src/args.c on server1. However, if the user is doing remote development from server2 (which also has an NFS mount of server1:/work/projects on /net/server1/export/work/projects), then yet another path mapping must be made.

For instance, if I do a remote build of Args_1 from my PC, the project thinks its project root directory is Z:/NetBeansProjects/Args_1. But the remote make command (run on server2) must be in /net/server1/export/work/projects/NetBeansProjects/Args_1. Subsequent running and debugging must also change path names. If my build fails, make will pass gcc error output for the file /net/server1/export/work/projects/NetBeansProjects/Args_1/src/args.c. For the IDE to open this on the client, this path must be converted to Z:/NetBeansProjects/Args_1/src/srgs.c.

We'll provide a path mapping table for you, and in many cases be able to fully populate it without help. If we do need help, we'll do as much as we can before asking for help.

Design TBD.

Miscellaneous Items

Server presence

We need server presence for:

  • Debugging support
  • Toolchain searches and validation
  • Tools for any functionality we can't handle with SSH

There are several options for doing this: installing binary executables, jar files or scripts. Most likely we will install scripts (shell or perhaps perl) when the user first defines a remote server.

TBD

Running applications with graphical UI

For NetBeans 6.5, you must explicitly set the DISPLAY property and have it display on a proper X server. We will not provide direct support for displaying the graphical program on the client.

Not logged in. Log in, Register

By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo