CndErrorHighlightingTestPlan

C/C++ Error Highlighting Test Plan

Accuracy tests

Common errors:

  • Misspelled IDs
  • Missed ";" (probably ";" after class is a special case?)
  • Unbalanced
    "()", "{}", "<>", "[]"

Projects:

"Real-life"

Projects list:

the same as for completion

How to test:

For a project

  • each file (or each N-th file, where N is ?)
  • erase each K-th ";"/brace (where K is a prime number, parameterized)
  • add brace after/before the same brace (M-th, where M is a prime number, parameterized)
  • we don't test unnecessary ";"

What can we (theoretically) check for error message

  • Existence of message
  • Number of messages
  • Placement
  • Message text

What we are going to check:

  • there is at least one message
  • placement of the 1-st one: only statistics
  • message count: only statistics
  •  ?? presence of the word (like "semicolon", etc) in the text?

Output details about each experiment to some simple format that

  • is more or less human readable
  • easy to find & sort

Inaccuracy tests

Testing Strategy

Check error highlighting on compilable configured projects. Each file in the project should be examined. Negative tests cases (containing code examples, on which compiler should fail) should be created for each project.

Tested projects

Use the same set of projects as for Model Accuracy testing. What is the list of projects? - quote, freeway, loki, clucene, litesql, Perl(?), Python(?), MySQL(?), ddd(?) (TBD

Scenario:

for each project:

  • open all files in editor
  • wait some time
  • count all 'C/C++ Error' annotations in each file
  • if file contains annotations then save full file name into html or text file
Not logged in. Log in, Register

By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo