ExecutionApiMerge

(Difference between revisions)
(How to do that [phejl])
(All requirements (not ordered))
Line 39: Line 39:
process
process
-
* parallel execution '''[phejl]''' Is there a problem with that? Why is it mentioned explicitelly?
+
* parallel execution '''[phejl]''' Is there a problem with that? Why is it mentioned explicitly?
* environment variables handling (initial get, ways to modify)
* environment variables handling (initial get, ways to modify)
* start / terminate processes
* start / terminate processes

Revision as of 13:51, 5 November 2012

Execution API merge

Contents

Problem statement

  • There is a need of common public API for executing external processes - either on local host or remote host.
  • There are several flavors of related APIs:
    • dlight.nativeexecution - non public (friend) API used by C/C++ modules and many modules in Studio ( 119 friends in total); All the code that uses this API is under our control;
    • extexecution - PUBLIC API for executing remote processes. It is plug-able and allows to pass own ProcessBuilder. As API is public, there are users of the API that are not under our control... (54 modules in nb codebase use it)
    • richexecution (within lib.terminalemulator)

dlight.nativeexecution is intensively used in CND. This module provides functionality for runnig processes on different platforms (*nix, windows, macos), it deals with processes starting/terminating, I/O, different terminals, signals, and so on. So it provides good functionality coverage, but has several big problems:

  • nowadays it requires serious cleanup;
  • it's implementation has many places where call to some method initiates a connection, which is unexpected in most cases
  • it is ssh-oriented. It's fundamental API object, ExecutionEnvironment, doesn't provide a way for specifying a way of accessing remote host
  • it's error-handling/flow is not well structured and not always clear

cleaned-up dlight.nativeexecution can be used as an implementation module with extexecution API, but extexecution should be extended in this case to support features proposed by the dlight.nativeexecution.

Scope ot the review

  • I would like to review an API (cleaned-up) of nativeexecution (not binded anyhow to extexecution) with the main aim to demostrate
    • what functionality this module provides (all this functionality was needed for CND's remote support)
    • how the functionality affected some API design decisions
  • Then I would like to make conclusions on what is missing in the extexecution (mostly this is about ExternalProcessBuilder), how it could be extended in a compatible way to make it possible for users to use full range of functionality provided by nativeexecution.

From here by term 'nativeexecution' I mean a new rewrittent API/implementation which is a result of clean-up of existent dlight.nativeexecution module. The possible way of dealing with modules (as I see it) is to:

  1. Deprecate dlight.nativeexecution
  2. Introduce nativeexecution (API/SPI/impl) [phejl] Maybe such API won't be needed afterall. Depends on how much can we integrate to extexecution.
  3. Extend extexecution and start using (2) as a implementation layer
  4. Move existent CND modules on the new APIs (extexecution/nativeexecution)
  5. Get rid of the 'old' nativeexecution.

Goal of the review

To provide an API (most likely extend extexecution) so that API users will be able to execute commands remotely and have access to all needed (as experience with CND shows) functionality.

All requirements (not ordered)

connection

  • connection initiation/cancellation/termination to a remote environment
  • connection error handling
  • track changes in connection state

process

  • parallel execution [phejl] Is there a problem with that? Why is it mentioned explicitly?
  • environment variables handling (initial get, ways to modify)
  • start / terminate processes
  • tracking process state changes
  • get process info (at least some - duration/start time etc..)
  • get process exit status (exit status or if terminated by signal - signal number; if core was dumped - need to know this)
  • signals sending
  • OS-neutral
  • helpers for handling special variable names (like LD_PRELOAD/DYLD_INSERT_LIBRARIES)
  • helpers to work with PATH-like variables
  • 'easy' processes start a.k.a ProcessUtils
  • encodings handling
  • pseudo-terminal, terminal I/O
  • internal/external terminals
  • unbuffering [phejl] What's this?
  • X11 forwarding
  • authentication - password/ssh key/agents
  • errors Handling / flow
  • info caching (?) [phejl] What's this?
  • shell commands support
  • ShellSession (a helper session for running small tasks (?))
  • trace-ability
  • I/O redirection

security

  • sensitive information protection
  • caches/temp files handling..

Additional possible usages that are NOT covered by CND/SolStudio use cases, but could be considered as well

  • 'Remote host' is not necessary a host

Here I mean that remote execution could be considered as an execution in 'other context'. Which could be, for exeample, running some command on a remote farm in the context of some ADE-view;

How to do that [phejl]

As this is pretty large task I think we have to come up with some basic plan. The target state should be extexecution module providing API with nativeexecution as implementation module. We will see what's doable.

The merge should be done upside down so we won't copy methods and classes at first place, but rather we will search for missing functionality and we will explore a ways how to add it to extexecution enriching it in a compatible way.

We should progress in steps starting from core functionality. This does not mean such steps should be visible in the VCS trunk though a named branch would be ideal solution for cooperation.

These would be necessary steps:

  1. Provide a core functionality in extexecution:
    • Connection handling
    • NativeProcess if needed
  2. Extend a functionality to handle things like:
    • Enviroment variables
    • Authentication
    • X11 Forwarding
    • ...
  3. Going further to cover all other requirements

Obviously this will need a bunch of iterations and will take some considerable amount of time before being commit ready.

Javadoc and code

Current version of execution javadoc and API code (mq patch) could be found: here.

Not logged in. Log in, Register

By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo