NetBeansDreamTeam/Proposals/Infrastructure

Contents

DRAFT: List of Infrastructure Problems/Solutions

Members of the NetBeans Dream Team have identified the following problems in the NetBeans infrastructure and suggests the solutions below.

The "DRAFT" list below comes from an e-mail received recently from Tom Wheeler, so instances of "I" mean Tom. Once other dream team members have added and commented, the draft status will change. The DRAFT text is simply there right now to indicate the list below is not complete yet.

Communication

Problem: (Tom) Changes to infrastructure are often poorly communicated (see Ivan/Emilian's recent thread on nbdev, which Wade forwarded to the list). This apparently even affects people inside of Oracle, and is much worse for people in the community since don't have access to intranet, company meetings, meeting developers in the cafeteria at lunchtime, etc. Additionally, changes to infrastructure seem to be made without regard to how it will affect work done by others. As an example, I have easily spent > 100 hours editing things in the FAQ. In the last five years, it moved from java.net Wiki to JSPWiki to Media Wiki and the Web-based mailing list archives have also recently changed. These changes have broken many links in the FAQ.

Proposed Solutions: Close communication with the dream team around how/when to communicate changes. The dream team should be the primary point of contact when there are changes, in order to discuss them and how to communicate them on a case by case basis. (Geertjan)

The poor communication about infrastructure changes affects a much wider audience than just the Dream Team. I think the solution is to handle infrastructure changes just like you do API changes; in other words, suggest the changes in a specific public e-mail list where any interested person can subscribe, be informed of these changes several days before they're made, ask questions and offer suggestions just as they could for API changes. (Tom)

Good idea. Should this mailing list be nbdiscuss? I would also be interested in the kinds of aspects of the NetBeans project that should be considered APIs (e.g, changes of wiki infrastructure is one, but what other things should be included here, can we formulate as specifically/generically as possible what aspects we are focused on here, i.e., when is there a need to communicate when changes occur?) (Geertjan)

Website Changes

Problem: (Tom) Nobody outside of Oracle can edit the www.netbeans.org Web site, aside from the Wiki (see my comments on nbdiscuss)

Further info needed: What kind of content on the site would be changeable by committers? What would the process be? What changes are dream team members suggesting it be possible to do by external committers?

Proposed Solutions: Investigate whether projects on netbeans.org can have external committers. E.g.. maybe platform.netbeans.org can have external committers who work on the tutorials. (Geertjan)

The above proposal is a step in the right direction, but sounds like it's still limited to certain parts of the site. I don't see the point of establishing arbitrary borders. What if there are errors on the main pages of the NetBeans Web site? Reporting them on issue tracker takes much longer than simply fixing them. If there are sacred pages which the community simply cannot be trusted to edit regardless of what level of experience they have, at the very least it should be possible to check out or download a read-only copy of the site's current content so one can directly make all the desired changes at once and then submit the patch with an issue requesting that they be fixed.

I believe for someone to be able to change something they must first follow some processes for a while and earn that trust. I think this is the way all projects I know about operate though the Wiki's are usually a different story. I think it is easier for that to happen if there are sub-projects one gains access to. For instance, if one knows the platform and Java code, but nothing about PHP, then why should they change the PHP stuff? With that being possible, then how does one lead a project and maintain the control needed to do so? I think borders must be in place. --Wade Chandler

What about reporting problems in netbeans.org to nbdiscuss? I.e., if you find a typo, report it to nbdiscuss. That's a simple process of sending an e-mail. If that process were in place, would you still want complete access to all of netbeans.org? What would be the purpose of that -- a question of principle or something that would actually be useful? (Geertjan)

Sometimes to slow and to much work to change a link, replace old pictures or add a new sentence. We don't need a full access to all pages (from my point of view), but I see so many pages with old content unresolved links and more. A fast change by a dream team member is (IMHO) useful.
Is it possible to transfer more an more page to a wiki structure (w/o showing the wiki path in the url)?
With the wiki infrastructure we have a good versioning system, we can see changes from users and can rollback unwanted/wrong content.
It should be possible to hide the page/discussion/edit/... wiki buttons in standard view? Only registered users with permission should see the wiki navigation. Then it looks looks like a normal Web page. my2cents --Arittner 06:22, 29 March 2011 (UTC)
I don't expect people to be able to change any part of the site they want without "earning" it. I think of meritocracy in other open source projects where you can't get commit rights until you've submitted enough good patches that the people in charge of the *module* decide to give you commit rights for that module. When I say module, I mean a subsystem or plugin unless the project is small. (Ryan de Laplante)

Mirror Sites

Problem: (Tom) No mirror sites for netbeans.org downloads (see my comments on this list). I have documented problems in which I was unable to download releases from the Akamai download system you use now, both from home and work. These problems are intermittent, but continue even now, and have prevented me from participating in 7.0 beta testing in any meaningful way.

Proposed Solutions:

Support and link to mirror sites as you once did and like most major open source projects do, at least for production releases. Such sites should geographically diverse (both physically and at the network level). They should be as simple as possible; i.e. just a server directory listing with links to files to download directly with HTTP (no Javascript, just keep it simple).

Cannot download older versions anymore

This issue is solved. Thanks for bringing it up! issue 198654.

Website Instability

Problem: (Tom) NetBeans web site is unstable (see #195839)

Proposed Solutions:

  • Don't make content management more complicated than it needs to be.
  • Test infrastructure changes in a staging area before putting them into production.
  • Revert infrastructure changes if they cause problems, then redeploy once those problems are fixed.

Mailing List Instability

Problem: (Tom) NetBeans mailing lists are unavailable for several hours. This has happened several times, but I've logged one such case (see #197873)

Proposed Solutions:

Proposed Solutions: Dump the current mailing list/forum system on netbeans.org in favor of Google Groups. (Tom)

Website Login Problems

Problem: (Tom) Many people experience problems logging into the NetBeans web site (see #197854 #186660).

This has been a known issue for nearly than a year. Although there is a workaround (deleting domain cookies and active login sessions), it is not well known and certainly not obvious to even experienced people, so I expect the majority of users just give up. Although this has been marked as a P1 issue and has been corroborated by Martin Fousek, Jan Pirek, Jesse Glick, Tom Wheeler and Tim Boudreau, the issue does not list a single attempt from the Kenai team to diagnose or fix the problem. Their only response (a day after the issue was initially filed) seems to be "we deployed a new version last night" and "lets see how it goes." Jan Pirek stated that the problem continues and asked on March 30 that they look into it, but they have been unresponsive.

Proposed Solutions: Make the Kenai team responsive and accountable to important problems or consider replacing NetBeans Web site infrastructure with something better.


Forums/Mailing List

Problem: (Tom) Integration between mailing lists and forums is poor (#196415). See also this thread, where another user pointed out the same problem. According to Charles Bedon, he contacted the NetBeans Webmaster (Jan Pirek?) a few months ago and was told that they don't have enough people to correct "these problems."

Proposed Solutions: Dump the current mailing list/forum system on netbeans.org in favor of Google Groups. (Tom)

Out of Office Notifications on Mailing Lists

Problem: (Tom) Annoying out of office notifications get sent to all subscribers of mailing list (#189253).

Proposed Solutions: Dump the current mailing list/forum system on netbeans.org in favor of Google Groups. (Tom)


Mailing List Search

Problem: Mailing list search feature is broken (#195966) (Tom)

Proposed Solutions: Dump the current mailing list/forum system on netbeans.org in favor of Google Groups. (Tom)

Mailing List Digests

Problem: There's no way to get a daily digest of a list. This makes it difficult to join lists with lots of traffic. (Antonio)

Proposed Solutions: Use a mailing-list mechanism that allows for digests. (Antonio)

Mailing List Latency

Problem: (Tom) Mailing lists seem to suffer from latency problems. There will be no messages for hours, then a dozen messages will all show up at once (i.e. within one minute of another). The message headers show messages having been delayed for several hours, though I cannot say defintively whether the problem is on NetBeans side or GMail (but I'd guess it is more likely NetBeans).

Further info needed: Is this a generally experienced problem? (Geertjan comment: I, for one, do not have this problem.)

I have experienced it dozens of times and Javier said that he experiences it as well. (Tom).

I had noticed that messages come in spurts into my work mailbox, but I always assumed it was an artifact of my setup (e.g. mailing list going into my gmail inbox, where with filters emails are forwarded to my work email account). --Akochnev

Proposed Solutions: Need to find more info about this before solutions can be proposed. (Geertjan)

Dump the current mailing list/forum system on netbeans.org in favor of Google Groups would fix it. I am subscribed to several other mailing lists through Google Groups and delays of several hours have never been an issue on those lists. Issue filed (http://netbeans.org/bugzilla/show_bug.cgi?id=197167) documenting a 14 hour delay. (Tom)

Anyone seen this happen with new java.net mailing lists (should be same infrastructure)? --Sreimers 15:39, 29 March 2011 (UTC)

Mailing List - Echo own posts

Problem: (Ryan de Laplante) I have never seen any of my own posts to the NBDT mailing list show up in my inbox. Originally someone at Sun looked into it and it seemed like I am the only person with this issue. Over the years I changed email addresses once or twice and continue to have this issue. It doesn't bother me too much, but it is worth mentioning. All messages I send to the mailing list are delivered to everyone else.

Proposed Solutions: None.

Top Level and Sub-Project Support

Problem:

Currently there is no ability for someone to bring logic or create it as a full sub-project within the NetBeans infrastructure and community. This limits the types of contributions which can take place out of the box. By out of the box I mean a top level or sub-project is created within the infrastructure, code is built, possible contributors are solicited, mailing lists, forums, source control, etc are setup and a sub-project, as a contribution to the NetBeans community, begins.

To me, this is where Eclipse is beating NetBeans as a community. Such a system allows for companies and individuals to come into an area within the community and establish a foot print and bind resources to the project. It is more of a stake of ownership. But too, this helps bring other contributions to the areas which those sub-projects depend upon such as the base platform.

It also gives people a place to house and build those things while also branding them within the NetBeans community as a whole. This gives a presence. As more do this, it not only gives them presence, but grows NetBeans, the project, the community, the support, user levels, resources, and gives NetBeans more presence as well.

Imagine this as top level projects with the possibility for sub-projects. Thus, there may be a top level project called PlatformX located at platformx.netbeans.org and then there may be platformx.netbeans.org/projects/centrallookup. Granted Central Lookup is probably to small to be in its own project, it is a stand alone idea, and serves at least as an example. It would probably be even nicer if projects could be accessed as sub-domains such as centrallookup.platformx.netbeans.org.


Proposed Solutions:

  • Can NetBeans.org (Kenai) support such features for a domain name or group of projects already? --Wade Chandler
  • If we do not know, then who do we need to be talking with regard to Kenai/Java.net? --Wade Chandler
  • How do Oracle folks feel about this in general? --Wade Chandler
  • Too, just like Apache, we need some concept of incubating. --Wade Chandler
  • Project leads need to be able to manage their own project or sub-domain and sub-projects. --Wade Chandler
+1, I like the idea with a kenai support for projects. --Arittner 06:36, 29 March 2011 (UTC)
+1, I like the idea with --Sreimers 15:39, 29 March 2011 (UTC)
  • Projects need
    • Sub-Domains (see the points on top-level vs sub projects in this section)
    • Ability to create sub-projects
      • How would that work with repositories?
      • Would a sub-project of a project use the same repository or a different one?
    • User and Permission Management Facilities
    • Mailing Lists
    • Forums
      • Integration with mailing lists would be nice
    • Web page management
      • Instead of large caching delays, it would be nice if there was some staging environment content developers could login to and see changes immediately. Then, the things everyone else sees could be accessed through the caches and a time delay for something to go live. Would make working on site pages easier to see them in action immediately.
    • Source Repositories
      • One for project web site
        • That or instead us a pattern such as X folder in your repository is your web site
      • One for actual source code
      • These need to be their own repositories separate from the giant single repository currently holding NetBeans
      • It would be nice if like the Eclipse projects, these would support multiple technologies such as SVN and Mercurial at the same time; perhaps git as well
    • CI environment
    • Ability to get built modules and clusters/suites into the correct AU centers
      • Please see the information on incubation for more info

Incubation

I currently propose we have a way to designate projects as incubating or not, and this determines how those projects show up in AU centers which people can register in their IDEs. For instance, there could be an incubator.dev, incubator.beta, and incubator.release set of AUs and then production.dev, production.beta, and production.release. Naming can be worked out later, but that is the general idea for how to manage what is incubating or not.

Projects, while incubating, will still show up as top level projects, but will be marked as such in the header for a warning for anyone viewing them. This should allow for a smoother transition for projects, and less for mentors and Oracle folks to do when something graduates from incubator to production.

Having some property which designates them as incubating allows for them to be able to be located in different search results or link sets. This means if one goes to some link, "Incubating Projects", they will possibly see categories and then projects in those categories and be able to click on them and go to their project pages and sub-domains. --Wade Chandler

Continous Integration

At the moment http://deadlock.netbeans.org is the server where CI is done for NetBeans IDE. Is it possible to find an "official" way to open this up (apply for being on CI), so that an important contribution can be build against trunk/releases automatically? --Sreimers 15:39, 29 March 2011 (UTC)

Yes, CI would be great. This is something Apache does for its projects, and it is good. It allows for building, testing, etc. Something different in NB land compared to most of Apache land is automated user testing. Are there any resources for shared visual VMs where AUT could be setup for NB sub-projects? If more clarity is needed on what that means, please let me know. --Wade Chandler

Project Leads and Management

I don't know how user management would work for the sub-projects of a top level project. Perhaps if you are a member of the top-level project, you have direct access to all sub-projects. That sounds reasonable to me. Thus, if you have administrator rights to the top-level project, you then have administrative rights to any sub-projects. --Wade Chandler

Mailing lists - Spammer blacklisting does not work

Problem: Mailing lists which are not moderated are often spammed and even when the spammer email is put into the blaclist in the list administration UI, it is not then blocked. (Opened P2, no response from kenai so far, # 198388)

Solution: Fix this feature.

Not logged in. Log in, Register

By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo