CNDModelAccuracy

Revision as of 15:07, 6 November 2009 by Admin (Talk | contribs)
(diff) ← Older revision | Current revision (diff) | Newer revision → (diff)

Accuracy



Definition

Accuracy means correct recognition of:

  1. Declarations/definitions (classes, templates, etc.) and their internal declarations/definitions
  2. Meaning of identifiers and operators - correct hyperlink to declarations/definitions and find usages

More formal accuracy definition


Measurement

Measurement subject

Code completion

Measurement of language construction recognition and meaning of identifiers

Find usages

Measurement of meaning of identifiers and operators

Necessary data providers

They should provide:

  1. Get declarations
  2. Hyperlink

(3. Find usages ())


Data necessary for accuracy measurement from point of view of compiler

Data providers

Elsa parser

C/C++ parser

Provides:

  • Links (file, line, column) from variables, functions, operators and qualifiers usages to their declarations

There are no links for type usages //only indirect links (through constructors and qualifiers)


Released under the BSD License

Gnu compiler

Provides:

  • Already transformed (optimized) AST
  • RTL (Register-Transfer Language) code
  • Optimized control flow graph
  • Debug information
  • Dwarf, etc.

Sun compiler

Provides:

  • WorkShop source browser information
  • Debug information
  • Dwarf, etc.

Sage++ (?)

Xogastan (-)

XML-Oriented Gcc Abstract Syntax Tree ANalyzer

C only

Provides:

  • GXL (Graph exchange language)

Cppx (-)

Gcc based parser

Based on Gcc 2.97 and 3.0

Open C++ (-)

Source to source translator

Provides:

  • AST as a PTree
  • Some semantic information

Have problems with templates, etc.

Other methods

  • Unresolved hyperlinks
  • Rename/Compile
  • Dwarf tests
  • Generate data for middle end or back end
  • Code completion tests



Current issues

Language

L1 - C/C++ language L2 - our grammar language

  • L1 != L2
  • !(L1<L2)
  • !(L1>L2)
  • L1/\L2 is parsed in a wrong way

Renderer

  • Renderer fails
  • Renderer fails on (L1/\L2)
  • Renderer fails on (L1\L2)

Time limits

  • Parsing of some code samples takes a lot of time

C++ grammar in not context free

  • Information from renderer is needed during parsing
  • templates
  • static checking

Macros

  • Parsing order
  • Grayed code
  • Doubled definitions
  • etc.

Measure of inaccuracy of accuracy measurement

N – some abstract code

Let distribution of errors in N is uniform


M – testing part of N

n = |N|

m = |M|

l – number of errors in N

p = l/n – probability of error in code


So probability of error number in M is x:

Error creating thumbnail: Image type not supported
If n->infinity
Error creating thumbnail: Image type not supported
(Poisson's distribution)


Probability of error number in M is x+t:

Error creating thumbnail: Image type not supported

x = pm

P = a ~ 99%

So from this equations we get ‘t’

t/m – measure of inaccuracy with ‘a’ probability


a = 99%, p = 10%

m = 1000 => t = 0,0253

m = 10000 => t = 0,0081

m = 20000 => t = 0,0057

m = 30000 => t = 0,0047

For normal distribution as a approximation of P(m), numbers are a little bit (10%) lower

Not logged in. Log in, Register

By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo