Mercurial Build Infrastructure Plan QA

According to migration of CVS sources into Hg and new mercurial build system infrastructure there were some concerns raised that would have to be solved sooner or later. The mentioned infrastructure separates the repository to Master repository (that should be always buildable) and the Preintegration one - each of them will have isolated build and test infrastructure system. This layout and structure of repositories should ensure the building process from the Master repository will be always successful and it should prevent of broken builds. On the other hand it brings following

Questions and concerns:

  • Marking issues in bug tracking system.
    Developer will mark issue as 'FIXED' while pushing his/her changes into Preintegration repository. It will be good to add new status - 'INTEGRATED' which will indicate that the fix successfully reached the Master repository and after that QE can mark the issue as 'VERIFIED'. The automatic notification process would be of great help to QA - when we can expect the build that will contain specific fix.
  • What builds shall we use for testing? The ones from Preintegration build system or the one from Master repository?
    The most of time QE will be focused on builds from central repository (MASTER). While e.g. in the HR mode we will have to use both: the build from Preintegration repository and then build from Master one.
  • What kind of tests do we expect to run at a specific level of each build system?
    With the respect to amount of time and resources we can hardly afford to execute full automatic and manual testing for builds from both Preintegration and Master build systems. First proposal for test execution and evaluation is:
  • For Preintegration build system:
    Junit tests provided by developers. (qa-functional tests sometimes fail and if the process is set to stop any pushing into Master repository in this case, there will have to be responsible person for test failures evaluating and deciding whether it's safe to push changesets into the Master repository or not.)
  • (jglick: developer unit tests sometimes fail, too. Need to remove all those which are dependent on threads, weak references, etc.)
  • For Master build system:
    all validation tests suites, junit, qa-functional test - stable - as for now. Also the most of manual testing will done on builds from the Master build system - used by the users and community.
  • Will there be used binary test distribution for both Master and Preintegration build systems?
  • (jglick: probably impossible, as developers routinely change both source and test at once in such a way that the previous test will fail, or not even link, against the new code.)


Not logged in. Log in, Register

By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo