akb-nal-blr-2013

7/30/2019 akb-nal-blr-2013

1/44

April 8, 2013 NAL, Bangalore 1

Experiences with Formal Verification in

the Design of High Integrity Embedded

System & DAE Initiatives

A.K. BhattacharjeeBARC

7/30/2019 akb-nal-blr-2013

2/44


Computing and Control

In DAE, some regulators feel it is not right

to use computer/ software for safety

systems of NPPs

Fear is built by computer world jargons like

Bug, Exceptions, NaN, RMA, hacking, cyber attacks

Their fear is also aided by media hyped

infamous computer systems failures

7/30/2019 akb-nal-blr-2013

3/44


Infamous Computer Caused Errors

July 28, 1962 -- Mariner I space probe

1982 Blast in Trans-Siberian Soviet gas pipeline

1985-1987 -- Therac-25 medical accelerator.

1988 -- Buffer overflow in Berkeley Unix finger daemon

1988-1996 -- Kerberos Random Number Generator

January 15, 1990 -- AT&T Network Outage (ESS-4) (DominoEffect)

1994 -- Intel Pentium floating point divide

1995/1996 -- The Ping of Death

1996 --Ariane 5 Flight 501..

1999 Mars Polar Landar

2010 Toyota Recall of 200,000 Prius(http://news.bbc.co.uk/2/hi/8505402.stm)
http://news.bbc.co.uk/2/hi/8505402.stmhttp://news.bbc.co.uk/2/hi/8505402.stm

7/30/2019 akb-nal-blr-2013

4/44


Subsystem 1980s 1990s 2000s

Propulsion 42% 38% 54%

Guidance and

navigation

6% 16% 4%

Electrical 6% 8% 8%

Operational

ordnance

2% 8% 0%

Structures 4% 6% 0%

Software and

computing

0% 8% 21%

Pneumatics and

hydraulics

4% 2% 0%

Unknown 37% 16% 13%

Source: DEVELOPING SAFETY-CRITICAL SOFTWARE REQUIREMENTS FOR COMMERCIAL REUSABLE

LAUNCH VEHICLES, FAA/NASA, 2006 (www.faa.gov)

7/30/2019 akb-nal-blr-2013

5/44


Evolution of Computer Application

in Indian NPPs

Data Acquisition and health monitoring ofhardware systems in Dhruva reactor at Trombayand FBTR at Kalpakkam (1980-85)

Safety related : Reactor power Control in 220MWe Reactor at Narora, UP, 1987

Safety: Reactor Protection in 220 MWe Reactorat Kakrapar, Gujrat 1989

All major Control, Monitoring, Test, Surveillanceand Operator Information functions in TAPP-3&4are computer based systems. 2004

7/30/2019 akb-nal-blr-2013

6/44


Late Eighties and early Nineties

I&C Designers started movingtowards application ofComputers

Qualification Issues

Lack of Standard Review

Process DAE Initiated IV&V practices

Initial Challenges inVerification of ReactorProtection System

Design reviews, CodeInspection

Verification ToolUnavailability

Massive manual efforts

Need for Safety Classification

Development of AERB GuidesD-1,D-10,D-20

Identification of SoftwareProcess Standards

Inputs from IEEE 7.4.3.2,IEC60880, US NRC

Development of AERB D-25

Development of In-houseStatic Analysis andVisualisation Tools (Assemblyand C) (1989-95)

Identified Thrust Areas inFormal Methods, V&V

7/30/2019 akb-nal-blr-2013

7/44


Issues with Computer based I&C

Systems

7/30/2019 akb-nal-blr-2013

8/44


Software & Hardware Caveats:

Technology & Regulatory Issues

Complex Requirements (SW & HW) How to check Inconsistencyof requirements?

Software Infinite State System

Difficult to test for Corner case bugs: How to reducedependency on low level testing ?(Effectiveness depends onHuman factor)

Internal States difficult to access : How to design for EffectiveMonitoring to make systems fail safe and resilient?

Software interfaces are conceptual: How to conduct EffectiveReviews?

Trustworthiness of Compiler & Operating System Output of a compiler is put onto the controller running a

RTOS

7/30/2019 akb-nal-blr-2013

9/44




Hardware Trustworthiness of Processors

Implementation of the Instruction Set Architecture (has it beenimplemented correctly?)

Robustness from security (Has it been evaluated from perspectives

of system security e.g. IEC62443?) Programmable Hardware Devices like FPGA Software Issues (Challenges & Issues with Design Process & Tools

are same as in software)

Device Failure : Can we monitor their internal states to predictfailure?

Analysis of Software Hardware Interactions (Can wemodel sw-hw interaction for analysing complexinteractions to validate behavioral and performanceaspects?)

7/30/2019 akb-nal-blr-2013

10/44




Security Vulnerabilities & Evaluation ofEmbedded Systems (Evaluation of

Architecture, Software, Hardware &Communication and Need for Security Plan)

Human factors in Design & Verification.What is the state of the Art?(e.g. IEC60880: Methods whichare applied purely manually are highly error prone. Therefore they shouldbe supported by tools which use mathematical techniques to reveal thestructure and internal function relationships of the software and to check for

internal consistency, consistency with some prior model,desirable/undesirable properties, etc)

7/30/2019 akb-nal-blr-2013

11/44


Software System Safety

Safety-Critical Software Functions are

those software functions failure of which

can directly or indirectly, in consort with

other system component behaviour orenvironmental conditions, contribute to the

existence of a hazardous state

7/30/2019 akb-nal-blr-2013

12/44


Safety and I&C Systems

When I&C systems perform functions

important to safety, these systems must

be demonstrated to be safe and reliable

with appropriate degree of confidence.

Safety critical functions must be identified

based on Postulated Initiating Events

(PIE)

7/30/2019 akb-nal-blr-2013

13/44


Generic Requirements of Software

Performing Safety Critical Functions

Upon detecting an anomaly or failures, the softwareshould remain in or revert to a safe state (RuntimeMonitoring?)

Override commands should require multiple

operator actions

The software should notify the controlling executiveduring or immediately after transiting to an unsafestate

This is in addition to the application logic Hence Requires a thorough Design Verification

7/30/2019 akb-nal-blr-2013

14/44


Calculation or computation errors (incorrectalgorithms, calculation overflow, etc.)

Data errors (out of range data, incorrect inputs, large

data rates, etc.)

Logic errors (improper or unexpected commands,failure to issue a command, etc.)

Interface errors (incorrect messaging, poor interface

layout and design, etc.)

Environment-related errors (improper use of tools,Compilers, changes in operating system, etc.)

Hardware-related errors (memory errors, SEUs,

unexpected computer shutdown, etc.)

Computation Errors

7/30/2019 akb-nal-blr-2013

15/44


Assessment

Evidence that the

software is correct (with respect to

specifications),

Safe (functionally, security)

completely implements the requirements.

In other words, the software in these

systems must be demonstrated to be safeand to have high level of integrity.

7/30/2019 akb-nal-blr-2013

16/44


Assessment

Integrity should be assured by developing

system/software using systematic,

technically appropriate, carefully

controlled, fully documented and review-able engineering process, which is suitably

interfaced with V and V activities.

Emphasis on Process

7/30/2019 akb-nal-blr-2013

17/44


Why we need Process Standards?

Assessment of software is guided by aregulatory standard. The documentaryevidence is recorded as a safety case.

Since dependability cannot be derivedconcretely from an assessment, we needan assurance on the development process.

Because we cannot demonstrate how wellwe have done, we demonstrate how hardwe tried Dr. John Rushby, SRI

7/30/2019 akb-nal-blr-2013

18/44


Formal methods in Design and Verification:

Industrial Initiatives in Safety Systems

The process for getting a formal proof for thecertification process is not well documented.European documents (such as IEC 61508)recognize formal methods/ proofs. Advanceshave been made in the area of formal methodsthat are now more practical and viable in "reallife" than when standards like IEC60880/DO178B were written.

DO178C has recommendation on formalmethods.

7/30/2019 akb-nal-blr-2013

19/44


IEC 61508 SIL

Safety-integrity : probability of a safety related

system satisfactorily performing the required

safety functions under all the stated conditions

within a stated period of time. Safety integrity level : discrete level (one out of a

possible four) for specifying the safety integrity

requirements... where SIL 4 has the highest

level of safety integrity and SIL 1 the lowest.

7/30/2019 akb-nal-blr-2013

20/44


IEC 61508, Risk Analysis and SIL

Risk analysis guides risk reduction.

By the allocation of development resources.

A Class 1 (Intolerable) risk usually

requires software designed to SIL3/4 (highest)

level.

A Class 2 (Undesirable) risk might

Require software designed to SIL2/3 levels.

Higher SILs require more resources

7/30/2019 akb-nal-blr-2013

21/44


Verification of Software Performing Safety

Functions

Architecture : Event Triggered/Time Triggered

Functional and performance requirements Functional Logic

Control flow, data flow, information flow, Timing

Interrupt handling and exceptions handling in embeddedsystems

Appropriateness of Finite Arithmetic, pointers and Bufferusage

Communication protocols Compliance to quality standards and programming

guidelines

Absence of malicious programs.

7/30/2019 akb-nal-blr-2013

22/44


TestingIs Complex

Dependability Assessment should not guided by Testing Alone

7/30/2019 akb-nal-blr-2013

23/44


Need for Rigorous and Precise Program Analysis

to detect data flow anomalies, RTE

Checking compliance to MisraC is not enough

7/30/2019 akb-nal-blr-2013

24/44


Need to think beyond Testing

Verification of High Level Requirements (HLR)

Arsenals: HL Modelling and Model Checking

(Statecharts, HMSC, Model checking, SCADE)

Verification of Low Level Requirements (LLR)

Arsenals: Verification of Code

(Assertion Checking Environment (ACE))

Verification of Absence of Runtime Errors (RTE)

Arsenals: Automated Rigorous Program Analysis (ACE-II,Astree)

But What about Tool Assessment?

7/30/2019 akb-nal-blr-2013

25/44

Assertion Checking Environment

Call Tree

readRTD applyLogic genSignals writeDevice

checkTrip

pre

{program}

post


HLR

LLR

LLR

LLR

7/30/2019 akb-nal-blr-2013

26/44


7/30/2019 akb-nal-blr-2013

27/44

Experience

Verification of ReactorTrip Logic

Verification of FunctionBlocks (~80) of an In-

house PLC Type safeness andverifying againstRuntime Errors

Translation Validation

Also used for externalprojects from ADA,VSSC, DRDL

Used PVS as

backend Theorem

Prover

Not Automatic anddifficult to be used by

Design Engineers


7/30/2019 akb-nal-blr-2013

28/44

Model based Design

Developed few controllers withSCADE and verified controllerproperties

If the loss_of_electric_load orturbine_trip signal is true andreactor power exceeds 20%

Full Power then AnticipatoryAction (AA) should start andshould get completed in timeT1+T2, where T1 and T2 are

predefined constants. AAlowers the operational set

point (OPSP) in proportion toreactor power based on thefollowing equation OPSP =NLSP - K * T


7/30/2019 akb-nal-blr-2013

29/44

Rigorous Program Analysis

Code is what runs and controls real world

Guarding against Run Time Errors due tomodular & finite precision arithmetic and

heaps Rigorous program analysis to detect

possible computation errors

Issues are with False positives and FalseNegatives

More precise analysis is required


7/30/2019 akb-nal-blr-2013

30/44


Field Programmable Devices

(FPGA)

HDL Design

Synthesis

Place & Route BitStream Generation

ManualProgramming

Requirement Specification

Simulation

Tool Certification

7/30/2019 akb-nal-blr-2013

31/44

Formal Verification Tool for VHDL Designs

(CFDVS-BARC)

Abstraction/Refinement

Manager

SymbolicSimulator

Verification

ConditionGenerator

ConstraintSolver

Property

Symbolic Constraints

Abstraction

RefinementHints

PropertyTranslator

IRTranslator

VHDLDesign

IR

TransitionRelation

PropertySatisfiedCounter

example

Achieves scalable verification of designs with both data and controldominated sub-parts

Combines symbol ic sim ulat ion, abstract ion-ref inement, and bounded

model checking w i th wo rd-level constra int solv ing

7/30/2019 akb-nal-blr-2013

32/44

Functional Verification of Hardware

Design

Used for the functional verification of VHDLdesigns used in in-house developed Hardwareboards used in C&I Applications

Properties in PSL extracted from FPGARequirements Specification and submitted forverification

Counterexamples produced by the tool helped inincreasing precision of specification

Papers in CAV'11 and TACAS'13


7/30/2019 akb-nal-blr-2013

33/44

Few Other Formal Verification

Projects

Verification of FIT Logic of ECCS for

Dhruva Reactor using Esterel, Auto and

Duration Calculus

Internal Communication Protocol by Spin

Object Code Verification for ADA (with

TIFR) Required large implementation


7/30/2019 akb-nal-blr-2013

34/44

Who will verify Tool?


7/30/2019 akb-nal-blr-2013

35/44


Tool Assessment

Independent Output Assessment: Can the outputof the tool be verified to be correct through anindependent means? Some possibilities includemanual review of tool output, comparison with a

second equivalent tool . Relevant History: Does the tool have a well-

documented history of usage where it hasconsistently produced acceptable results? The

history of usage may include similar applications.

7/30/2019 akb-nal-blr-2013

36/44


Will the tools output be verifiedas per applicable standard?

Can tool insert error in

software?

Are processes of standard

Eliminated, reduced orautomated by use of tool?

No Qualification

Necessary

Tool must

be Qualified

No

Yes

Yes

No

Yes

No

Tool Qualification Requirement

Who

will

do?

7/30/2019 akb-nal-blr-2013

37/44

Some Discussions


7/30/2019 akb-nal-blr-2013

38/44


Effort Reduction for Better Design

Review Better Models of higher level specification

Data Flow Equations

State Transition Diagrams

Hierarchical Designs

Invest efforts in validating automated codegenerators

Verify once and use many

Reduce Efforts in Programming and Unit Testing

Admissibility of Regulating Norms to be seen

Promote Component based Designs

Reuse Verified components with due care of environmentand rigorous traceabilty.

Can be

reviewed

by domain

experts

7/30/2019 akb-nal-blr-2013

39/44


Promote Reviewable System

Design

Design Tools: Domain SpecificModelling Language,

Automatic code generator (Verify Once) Proof obligation generator (For Safety

Functions)

Analytical Tools: Compliance analyzer (Rigorous Program

Analysis)

Formal proof generator (For SafetyFunctions)

Security Properties

7/30/2019 akb-nal-blr-2013

40/44


New Issues

Can we be future ready?

Distributed Heterogeneous System

Handling Multicore processors

Failsafe Programmable FPGA designs

Languages

Model based, Type safe languages, Functional

Compilation issues

Rigorous Program Analysis

Vulnerabilities on modern architectures

7/30/2019 akb-nal-blr-2013

41/44


How to convince end user?

Formal proo f is (significantly but not

exclusively) about showing you got it right (and

hence part of the argument for dependability

claims) but, even then, proof is just formalreasoning about some properties of the

specification and design, and proofs rarely

compel a sceptic to become a believer. So

proofs are more important to the engineer thanto the customer or public. Prof. Bev Littlewood, UK,Safety Expert

7/30/2019 akb-nal-blr-2013

42/44


Soundness For Formal Methods?

Of the formalism

Of the software tools

Of Axioms and assumptions

How to convince Certification Authorities

What should be provided to certification authoritiesabout soundness? Operational Semantics, Proofsystem, Theories of bit vectors? How long the prooflasts!

If I do not understand your equations, I may not

certifyFormal Methods inspired Debugging and Testing to

build confidence

DAE C t ib ti th h E t l

7/30/2019 akb-nal-blr-2013

43/44


DAE Contributions through Extramural

Research

Setting up of CFDVS at IIT Bombay to

promote research in Formal Verification

Techniques.

Development of Tools and Techniques

with improved precision and scalability in

collaboration with CFDVS.

Development of Automated ProgramTesting Techniques at IIT Kanpur.

7/30/2019 akb-nal-blr-2013

44/44


Thank You

Documents

akb-nal-blr-2013