To build test distribution is very simple now. You only need to checkout CVS files for a NetBeans dev build, build IDE and test distribution:
cvs co standard_nowww ant -f nbbuild/build.xml ant -f xtest/build.xml ant -f nbbuild/build.xml build-test-dist -Dtest.fail.on.error=false
Ant needs more memory for building tests than is default value. So increase it by overriding ANT_OPTS environment variable.
ANT_OPTS=-Xmx512m
The property test.fail.on.error ignore compilation failures because few tests are still uncompilable. It is curious that some test are uncompilable. Why developer write tests if the test is uncompilable after a while. IMHO all the tests will have to be run daily if continuos integration is dream. I hope it will be solved as soon as possible.
The tests are built into nbbuild/build/testdist folder and packed to nbbuild/build/testdist.zip file. In the root folder of test distribution is short README.txt file. It contains description how to run tests. For example when you want to run all test(qa-functional and unit), you need only run ant :
ant all -Dnetbeans.dest.dir="${netbeans.home}"
The netbeans.dest.dir property contains directory with NetBeans installation. Te result of testrun is formated to html files. The html files are available in:
More details about binary test distribution are here.
Wondering whether tests are really uncompilable or if it's just your mistake? Check out continuous build results. You can also get the latest testdist.zip from trunk CVS sources.