cornercorner
FeaturesPluginsDocs & SupportCommunityPartners

SpeedWayTestPlan

Speedway Test Plan

1. Introduction

This test plan covers both black box testing of web based application for cloud access and grey box testing of API provided for cloud access performed by QE. The scope of the plan is for public release of the Speedway project. Target is stable and useful good quality product available on public website.

1.1. Objectives

  • To detail activities required to prepare and perform QE testing of SpeedWay UI.
  • To detail activities required to prepare and perform QE testing of SpeedWay API.
  • To communicate to the respective parties those activities that they are to perform and the schedule to be followed.
  • To define the environment needed to test SpeedWay UI including bug tracking system, dedicated testing servers etc.
  • To define risks.

1.2. Quality criteria

  • No P1 -- all P1s fixed
  • P2 fixed or waived.

2. Tested areas

Tested areas are listed below. The things not listed will not be tested at the time of this release. The areas are listed as they appeared in standard workflow out of order of priority.

2.1. Speedway Web UI application

2.1.1. Defining testing

  • Will be covered by QA team. Resp.: Michael Nazarov
  • General testing
  • UI elements' alignment testing.
  • UI elements' availability and functionality.
  • List of testing UI items for current version of application. (TBD)
  • UI testing
  • Login using Sun ID
  • Login using SDN ID
  • Files ystem testing
  • Upload from Desktop
    • Upload to default root
    • Upload to subdir(s)
    • Download from (Kenai)
      • SVN
      • CVS
      • Hg
      • GIT
      • Download to default root
      • Download to subdirs
    • Execute Actions:
      • Rename
      • Create Subfolder
      • Delete
      • Add Files
      • Download
    • Upload (just point at "root" dir and auto-tar it up)
  • Machine
    • Select M3000 or T2Plus
    • Release machine
      • Does it cleanup SGD
  • Change projects
    • Does it release machines
    • Does it cleanup SGD
  • SetUp?
    • Build configurations: change/set defaults
    • Configure
    • Clean
    • Build
    • Test
    • Install
  • SGD Applications
    • xTerm
    • Desktop
    • NetBeans?
    • SunStudio?
  • AJAXterm/WebTerm

2.1.2. Methodology

Mostly manual testing. Some automation is possible for underlying layer using direct HTTP calls and for UI using recording system like Selenium. Complete compatibility of such system with UI is not known for sure. Some actions required image analysis because existing UI uses different images without textual definitions to report state. Should be discussed.

2.1.3. Resources

1 day per OS/Browser pair.

2.2. Scalability & Network testing

2.2.1. Defining Testing

  • Will be covered by QA team. Resp.: Michael Nazarov
  • Response time & availability when application is under concurrent usage.
  • Network availability of the application

2.2.2. Target Testing

  • Network failures.
  • Slow connections.
  • Machine switching.
  • Simultaneous connection from different hosts.

2.2.3. Methodology

Ensure application works through proxy as well as using direct connect. Some testing like physical cable disconnection is very hard to automate, so manual checking will be used. Simulated concurrent access by 10, 25, 50 users will be done automatically using some available tools.

2.3. Performance

Metrics and boundaries are subject to discuss at the broad team. We can focus on web response or on performance of each particular service on the cloud. Needs to be defined an order of priorities for this. This testing can be skipped if time is needed to do functional and scalability testing.

2.3.1. Defining Testing

  • Service performance for each main service available measured on server side.
  • Browser responsiveness.

2.3.2 Target testing

  • Stress testing.

Such testing is subject for automated testing using API layer, not UI. UI testing with efforts of many number of user is also possible but seems to be impractical. In addition stable version of environment required for such testing for sure, no performance testing before appearing of stable (in all meanings including stable work and UI itself finally defined and implemented) and useful system.

3. OS/Browser matrix (TBD)

Testing will be done mainly using Firefox 3 as this is available for all platforms we care about.

WinXP Linux Open Solaris 2008.11 Open Solaris 2009.06 MacOS x86 Solaris 10 Windows Vista
IE 8  ? N/A N/A N/A N/A N/A  ?
Mozilla Firefox 2.0 + + + + + + +
Mozilla Firefox 3.0 + + + + + + +
Safari 3 N/A N/A N/A N/A + N/A N/A


? - if time permits. Minimal user display resolution to test on will be TBD to assure main page is correctly displayed.

3.1 Methodology

HTML rendering is subject to manual testing anyway. There is no (practical) way to detect was page showed correctly or no. There is way to automate page functionality like checkboxes behavior etc.

4. Scenario testing

  • Will be covered by QA team.

QE needs to get more detailed scenario for Speedway usage at demos at J1. Also for regular test specification QE will define several top scenarios which will be used at sanity testing during different milestones,...

4.1 Target Testing

4.2. Methodology

Ensure typical workflow goes smooth and natural way.

4.3. Resources

Depends on scenario complexity for each OS/Browser/Scenario triplet.

5. Bugtracking & Daily Builds

Issues should be filed and tracked using Bugster. NO WIKI "TRACKING"!

Internal buids availability

Internal builds availability and their quality is the main dependency QE has on Development team. Close to the 1st public release of Speedway QE needs to have a live and staging instance of Speedway servers. Staging will be used for testing of the service before it goes public. Based on current scope of the work QE assumes around 5 working days will be needed to move build from staging to live server. In a case of just selected bug fixes it can be much less. This time estimate can change as the product will grow and processes around it will be more mature. In time between milestones during regular development QE expects a daily build of speedway on staging server daily or at different agreed on interval.

Testing intervals on development builds - TBD.

6. Resources allocation

Note most of resources calculated for single bundle. Each new bundle requires same amount of time actually.

Item Defining Targeted
DCT UI application
Network
Branding
Performance
Compatibility
Display
OS/Browser
Scenarios
*TOTAL* TBD TBD


---

7. Schedule

Relates to overall project schedule. Detailed QE schedule will be done after P-team discussion.

8. Risks

8.1. Bugtracking system

  • Risk: Bugtracking system not yet defined. Developers actively refuse to use Bugster and prefer to use plain text on Wiki without any functionality to real tracking.
  • Contingency: force developers to use Bugster and track filed bugs.

8.2. Staff resources

  • Risk: manual testing required for some areas (<complete> automation is not possible).
  • Contingency: manual testing with all possible help from other groups. Hiring interns for limited time.
  • Risk: huge matrix of browsers supporting (<complete> automation is not practical).
  • Contingency: dropping some of configurations. Hiring interns for limited time. Manual testing with all possible help from other groups.

8.3. HW resources

  • Risk: limited access to some lab hardware.
  • Contingency: getting unlimited access for existing HW or new HW for DCT team only required. Example: Solaris Sparc based machine.
  • Risk: testing server maintain.
  • Contingency: use local server (impossible?) or freeze appropriate testing till server with correct and workable content appear.

8.4. UI

  • Risk: Testing UI is still subject to change. It is in development by standalone team and QA have no any contacts with.
  • Contingency: Force finalizing UI. Provide useful way for QA team to communicate with UI developers.

8.5. Program/Project

  • Risk: Scheduling of project is still subject to discuss.
  • Contingency: trying to get complete and clear schedule from architects. Until then we don't give guaranties about testing quality and coverage.

8.6. Other groups cooperation

  • Risk: No complete list of contact persons defined for UI problems, schedule, HW etc.
  • Contingency: Force creating such list or define special QA mailing list.

8.7. Documentation

  • Risk: No points to existing specifications of subject to test.
  • Contingency: Make additional investigation of UI functionality and purpose.

Related links

Approvals

Date Approver Notes


History

Date Person Version Notes
Jul 2, 2009 Michael A. Nazarov 0.2 Updates according to new schedules from developers.
Apr 13, 2009 Michael A. Nazarov 0.1 Initial creation.
Retrieved from "http://wiki.netbeans.org/SpeedWayTestPlan"