GrantProjectsEvaluationGuidelines

NetBeans Innovators Grant Projects Evaluation Guidelines

This page contains guidelines for evaluation of 13 completed NetBeans Innovators Grant projects by NetBeans Dream Team members. This testing will happen from August 11th to August 24th and will be divided into 2 rounds. The two rounds will serve for judging if projects deserve their grants and Gold and Silver Awards will be selected from the scores created during these reviews.

[[{TableOfContentsTitle=TableOfContents} | {TableOfContents title='Table of Contents'}]]

Voting Rules

  • each plugin is reviewed by at least 3 DT members
  • reviewers rate each plugin with Yes/No i.e. deserves the grant?
  • reviewers score each plugin which is complete (Yes) with a 1-10 score using the evaluation criteria listed below.

Evaluation Criteria

There are 2 overall things being evaluated and determined.

  1. Whether the project is complete and should be paid its initial grant money. All projects are eligible for their initial grant money as long as they are complete per the owners submitted proposal. Please see the proposals.
  2. Which 4 projects are better or rank higher than all other projects, and which 2 are better out of the best 4. This will determine Gold and Silver awards of which there are 2 each. Large and small projects are eligible for Gold and Silver awards. Projects are evaluated on the ability for new features to be added, their coolness, or how innovative they are and how much enthusiasm they may build around NB, if the project owner and team have a plan and list of things to do to the project in the future, and their quality and usability compared to the other projects.

Evaluation for completeness and if the grantee is paid initial grant money

If a project does not meet the following criteria it is not eligible for the initial grant money, nor is it eligible to compete for a Gold or Silver award.

  1. project quality is acceptable (no obvious showstoppers)
  2. project usability is acceptable
  3. project has acceptable documentation
  4. project accomplishes project owners proposed goals from their NB Innovators Grant proposal
  5. NB compatible license - must be free and open source
  6. module works on 6.1, 6.5m1, or most recent 6.5 daily build
  7. does the module follow the rules stated in http://www.netbeans.org/grant/innovators-grant-proposal.pdf

Evaluation for Gold and Silver Awards

The following will be used to give a 1-10 score to each module. Each of the criteria have a weight of 2 with a possible value range of 0-2. Dream Team members should use this criteria to reach a score for modules they evaluate. Once an overall value is determined by a member/judge, then a rounded value should be derived by rounding up. i.e. 7.25 = 7.5, and rounding should not take place until the you have totaled the points. Rounding is done to make polling easier (not as many options).

Break each section up into 1/4 of a point. This gives a range of .25-2 for each area of criteria. Once you have assessed a score between .25-2 for each criterion add them up and it will give your total score. Thus 2+2+2+2+2 is a perfect 10 and .5+.5+.5+.5+.25 is a 2.25 which gets rounded up to a 2.5. As we allow half a point, we are rounding up to the half versus rounding down.

  1. overall, how useful and cool or how much interest can the module generate for NB
  2. project can be enhanced and extended in the future, or it is an idea and design which makes it more possible
  3. project owner has a plan and ideas exist to move this project passed the initial grant
  4. project quality (is it better quality than others) - documentation exists and its quality, source code follows NB conventions, tests exist (maybe they exist in one and not the other)
  5. project usability (does it work better than others from a usability perspective) - do you have to think too much about to use it or is it obvious


1st Round (8/11 - 8/14)

Plugins to be reviewed:

Project Name DT Reviewer DT Reviewer DT Reviewer
CashForward future plans Wade Chandler Alex Kotchnev Kristian Rink
Cube°n future plans Sven Reimers Masoud Kalali Ryan de Laplante
JavaSpaces future plans Tim Dudgeon Sven Reimers Vy Ho
MONOH future plans (see todo) Wade Chandler RichUnger TomWheeler
Numbered Bookmarks future plans Antonio Vieiro Ryan de Laplante Wade Chandler
PDFViewer future plans Antonio Vieiro Ryan de Laplante Varun Nischal


2nd Round (8/15 - 8/19)

Plugins to be reviewed:

Project Name DT Reviewer DT Reviewer DT Reviewer
IvyBeans future plans Sven Reimers Masoud Kalali Wade Chandler
NB Project future plans (see future plans section) Antonio Vieiro Tom Wheeler Tim Dudgeon
NetBeans Spot/Sun SPOT Plugin future plans (see "Next Steps" near top) Kristian Rink TomWheeler Vy Ho
Scala Support future plans Sven Reimers Varun Nischal Masoud Kalali
PL/SQL Editor future plans Masoud Kalali Wade Chandler Rich Unger
Project Darkstar Tools and Mobility Support future plans (see Future Releases) Wade Chandler Alex Kotchnev Vy Ho
Resource Bundle Editor future plans (see Planned Features) Sven Reimers Ryan de Laplante Varun Nischal


Not logged in. Log in, Register

By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo