Application Note 4 Independent Safety Assessment - … Iss 2.pdf · Application Note 4 . Independent Safety Assessment . ... Application Note 4 – Independent Safety Assessment

Engineering Safety Management

Yellow Book 4

Application Note 4 Independent Safety Assessment Issue 2.0

Disclaimer RSSB has taken the trouble to make sure that this document is accurate and useful, but it is only a guide. Its content does not supplement nor remove any statutory, common law, fiduciary, regulatory or contractual function,

Application Note 4 – Independent Safety Assessment (Issue 1.0) Page 1

duty or obligation owed by other persons. In issuing this document, Rail Safety and Standards Board Ltd ("RSSB") makes no warranties, express or implied, that compliance with all or any documents published by RSSB is sufficient to ensure safe systems of work or operation, nor does RSSB undertake to be responsible for any monitoring or compliance with its recommendations or any duties or obligations owed by other persons. Persons wishing to adopt the recommendations should seek independent advice on the possible consequences before doing so.

Uncontrolled When Printed Document withdrawn as of 01/01/2012

Roles in each system lifecycle phase Appendix A

Published by RSSB on behalf of the UK rail industry



Contents 1 Introduction 4

2 Why Have Independent Safety Assessment? 5

3 Application Note Stakeholders 6 3.1 Yellow Book 3 Stakeholders 6 3.2 ISA Stakeholders 6

4 Customer and Project 9 4.1 Is there a need for an ISA? 9 4.2 Selecting an ISA 10 4.3 Working with an ISA 12

5 ISA Organisation 15 5.1 Demonstrate 15 5.2 Manage 17

6 ISA Team and ISA Engineer 19 6.1 Planning 19 6.2 Delivering 22 6.3 Reporting 26

7 Safety Authority 27 7.1 Safety Authority ISA Framework 27 7.2 Accepting a Change 28

8 Abbreviations 29

9 Acknowledgements 30

10 Documents Referenced 31

A Appendix: Roles in each System Lifecycle Phase 32

B Appendix: Competence Management Resources 37

C Appendix: ISA Remit Checklist 38

D Appendix: Cross Acceptance 39

E Appendix: Examples of ISA Methodologies 42



Introduction Section 1

1 INTRODUCTION The Yellow Book Steering Group has decided to supplement Yellow Book 3 [1] with a series of Application Notes. Each note provides more detailed guidance on a particular aspect of the Yellow Book. This document is an Application Note for Independent Safety Assessment. It provides guidance on key aspects of Independent Safety Assessment for all stakeholders involved with Independent Safety Assessment activities.

Volume 1 of the Yellow Book [1] contains an Engineering Safety Management (ESM) fundamental on Independent Professional Review (IPR) it is supplemented in section 14 of Volume 2 with guidance for Project Managers and independent assessors/auditors. The term IPR embraces both Independent Safety Assessment and independent safety audit activities, but in practice the two overlap. An ISA may well undertake both types of activities on a single project, and this Application Note recognises that this is frequently the case.

This Application Note should be read in conjunction with the Yellow Book [1] as it supplements the guidance already given. It provides guidance for each of the stakeholders involved with Independent Safety Assessment, defining their roles, responsibilities and inter-relationships.

To support the stakeholder-specific guidance, Appendix A provides a condensed summary of typical Independent Safety Assessment-related tasks from a system lifecycle perspective. It complements the main body of guidance by providing practical examples from a typical Independent Safety Assessment project.

We have endeavoured to make this Application Note accurate and useful, but it is only a guide and it is not intended to be used prescriptively. We have focused on those aspects of Independent Safety Assessment which in our experience, need further guidance, rather than attempting to address all aspects of the subject. We do not give any form of guarantee that following the recommendations in this Application Note will, on its own, be enough to ensure an acceptable level of safety.

We are continually working to improve the Yellow Book [1] and we welcome comments. Please contact us at the address below, if you have a suggestion for improvement. ESM Administrator Rail Safety and Standards Board Evergreen House 160 Euston Road London NW1 2DX Phone: +44 (0)20 7904 7777 Fax: +44 (0)20 7557 9072

Or you may email your comments [email protected]



Why have Independent Safety Assessment? Section 2

2 WHY HAVE INDEPENDENT SAFETY ASSESSMENT? Independent Safety Assessment is a tool for projects to use to help ensure that a desired change is specified and implemented correctly. The greater the risk of death or injury if the change is not specified or implemented correctly, then the greater the importance of a systematic and rigorous approach to Independent Safety Assessment.

Consequently, best practice ESM as detailed in the Yellow Book [1] recommends that Independent Safety Assessment is conducted with a level of rigour and independence that is related to the degree of safety criticality of the change.

All parties involved in making safety-related changes have a legal duty to guard against negligence. This duty is discharged, in part at least, by following established good practice. Independent Safety Assessment is regarded as good practice and therefore provides additional protection against negligence.

The following guidance has been written on the basis that the ISA is required to be fully independent of the project, i.e. a separate organisation. The guidance will, therefore, need to be tailored in circumstances where the required level of independence is required to be less stringent.



Application Note stakeholders Section 3

3 APPLICATION NOTE STAKEHOLDERS

3.1 Yellow Book 3 Stakeholders The Yellow Book 3 [1] is based on four different roles or stakeholders:

• Customer (initiates the requirement for change, then operates or uses the results of a change);

• Project (engineers the change and controls the risk);

• ISA (reviews the change and assures that the risk associated with the change is as low as reasonably practicable (ALARP)); and

• Safety Authority (accepts the change and its associated risk);

There are various interpretations of the term Safety Authority, and readers should recognise the differences and tailor the guidance to match their own situation. This Application Note uses the definition above, although the following definitions are also relevant (not all of the guidance will be applicable if the Safety Authority is operating in accordance with one of these definitions):

⎯ Accepting or Certifying Safety Authority – Equivalent to definition of ‘Safety Authority’ above. Often the Infrastructure Controller fulfils this role.

⎯ Yellow Book Safety Authority – As defined in the Yellow Book [1] this role is to promote ESM within an organisation, and to ensure that work produced by the organisation meets required safety standards.

⎯ Supervisory Authority – In each EU member state, this body is responsible for authorisation of placing into service those systems that are within the scope of the European Railways Interoperability Directives [6] and associated Technical Interoperability Specifications (TSIs). In the UK this body is the Health and Safety Executive (HSE).

⎯ Governmental Authority – this is a government body that regulates, inspects and accepts changes to the Railway.

3.2 ISA Stakeholders For practical purposes, and to reflect reality in this Application Note we will refine the ISA role as comprising:

• ISA Organisation (legal entity which offers ISA services);

• ISA Team (provides professional opinion on the acceptability of the risk, related to the change, and is part of the ISA organisation); and

• ISA Engineer1 (individual who carries out specific Independent Safety Assessment activities as part of the ISA Team).

1 The term engineer is used here and throughout the Application Note, it is recognised that while an assessor is normally an engineer that there are also non-engineering assessors.



Application Note stakeholders

3.2.1 Notified Body as an Independent Safety Assessment Stakeholder

One further role is relevant in the context of this Application Note as a result of changes in legislation which affect all European railways:

• Notified Body (verifies that change conforms to applicable standards).

The diagram below provides a high level overview of the relationships between the roles defined above. This is representative of the scenario where a Notified Body (NoBo) is required and therefore appointed to a project.



Application Note stakeholders Section 3

Customer

SafetyAuthority

Project

ISAOrganisation

Team

Need to makea change

Initiate project toengineer the change

Agree terms of reference(TOR) for project

ProposeAcceptance Criteria

Endorse AcceptanceCriteria

Appoint ISA

Need NoBo?YN

Agree ISA Remit

NotifiedBody

Appoint Lead ISA,Team or Engineeras appropriate

Appoint NoBo

Agree NoBoTOR

Kick off & planISA project

Requirements/SpecificationPhase

Monitor ISAProject

Design Phase

Implementation/BuildPhase

Monitor ISAProjectMonitor ISAProject

Testing/TrialsStaged Acceptance,Review AssessmentReport for Stage

Final SubmissionFinal Acceptance,Review FinalAssessment Report

Monitor ISAProject

Monitor ISAProject

Handover

Operate/UseChange

Check ProjectComplianceCheck ProjectComplianceCheck ProjectCompliance

Check ProjectCompliance

Check safety compliance

Complete TechnicalConformance File & NoBoReport

CertifyConformance

HandoverCertification

Gain Supervisory AuthorityAcceptance of Certificate

Activity which starts another role

Activity requiring interaction with another role

LEGEND

Activity

Assess/Audit

Assess/Audit

Assess/Audit

Assess/Audit & AssessmentReport for each Stage

Final Report,Recommendations

Figure 1: Illustrative Interactions Between Roles (for a Generic Change Project)



Customer and Project Section 4

4 CUSTOMER AND PROJECT The Customer can be defined as the entity that pays for Independent Safety Assessment services and the Project can be defined as the entity delivering the change that is to be assessed.

The Customer and the Project can be, but are not necessarily, the same entity. For instance, where a supplier develops a new product (such as an interlocking, which is part of the railway infrastructure), and wishes to obtain product approval, the supplier is both the Project, which is engineering the product, and the Customer for the Independent Safety Assessment of the product. By contrast, if a new type of train is being built, the Train Operator may be the Customer, but the Project may be implemented by the rolling stock manufacturer.

Concept & Feasibility RequirementsDefinition

Design &Implementation Installation & Handover Operations &

MaintenanceDecommissioning &

Disposal

Communication and Interaction

Need an ISA? Working with an ISA

Monitoring ISA independence & competence

ISA observations and reports

Recognising the boundaries

Roles and responsibilities of the Project Team

Selecting an ISAISA remitISA selection

Figure 2: Roles and Responsibilities of a Customer or Project (within EN 50126 Lifecycle2)

4.1 Is there a need for an ISA? Customers and Projects should use the guidance presented by the Yellow Book [1] in section 14.4.1 to establish whether or not an ISA is required. If an ISA is required, then the Customer or Project should determine whether the ISA should be internal or external to the Project’s organisation. This should be decided based on Yellow Book [1] Table 14-1, which indicates the required level of ISA independence in relation to the safety criticality of a proposed change.

The Project should seek advice from the Safety Authority to confirm that the proposed degree of independence is acceptable.

The following guidance addresses ISA issues regardless of whether the ISA is internal or external to the Project’s organisation. In the situation where a project or customer uses an internal ISA then issues such as Commercial Confidence may become irrelevant.

2 Lifecycle depicted is an abstraction of EN 50126, the mapping is provided by the Yellow Book [1] in Table 11-1.




4.2 Selecting an ISA

4.2.1 ISA Remit Before starting to compile a remit, projects should consider the following:

• Safety Acceptance Criteria. Projects should establish the fundamental criteria by which the acceptability of safety risk will be judged before employing an ISA – most importantly whether it is a risk-based approach (such as ALARP) or a prescriptive approach based on compliance with standards. Clarity in this area will ensure that the ISA’s effort is focused towards the correct objective. Projects should, after appointment of the ISA, ask the ISA to comment on the proposed safety acceptance criteria to confirm their suitability and sufficiency, unless of course the criteria have already been accepted by the Safety Authority.

• The role of the ISA on the project. The ISA cannot act as an advisor to the Project or perform tasks such as safety engineering (see section 4.3.4 for a definition of advice). However, the ISA can help by planning the assessment activities in a way that assists the Project’s management of the safety risks (and consequently the project delivery risks), and should be encouraged to do so.

• The role of the NoBo on the project. The Project needs to determine if a NoBo is required. If one is required then the remit for the NoBo should be written in conjunction with the ISA remit. The remits should ensure optimal use of effort, and adequate provision of time and resources to facilitate the interaction of all parties (Project, ISA and NoBo) without duplication of effort. The Independent Safety Assessment output should be targeted to meet the ISA remit, the Safety Authority’s requirements and also the NoBo’s requirements.

The roles of an ISA and a NoBo appear to be similar but they are subtly different. An ISA is appointed to review a submission and provide observations as to the completeness and robustness of the safety arguments presented. A NoBo is required to assess whether the Essential Requirements for interoperability have been met according to the criteria specified in the TSI and Notified National Technical Rules (if any). As interoperability is wider than just safety (it includes matters such as reliability, maintenance, environmental protection and health as well), the NoBo role is different. In addition, a NoBo is not required to give observations on the degree of compliance. Either an item complies with the Standard or it does not.

In some situations the NoBo and the ISA are the same organisation whereas in others they are distinct. At the time of writing this Application Note, there is insufficient experience of the role of NoBo to determine which approach, if any, is preferable.

To produce a comprehensive remit the Project should use the structured checklist provided in Appendix C. This will guard against omissions and assist in anticipating future requirements – but it is not exhaustive. For each item on the checklist, the Project should request that the prospective ISA demonstrates understanding of, compliance with, each of the items within the context of the Project. The most important, but often missed, remit information is:




• Project Context. If the Project is part of a larger programme of work, the context of the project within the overall programme should be provided, detailing the hierarchy both above and below the Project. The context should include the approvals processes and technical scope. Organisational structure of the Project should also be provided, detailing roles and responsibilities.

• ISA Support. An ISA can be used to support a project in various ways. For instance, in respect of the Safety Plan the ISA could support that the processes detailed in the Safety Plan were satisfactorily carried out. Alternatively, or indeed additionally, the ISA could support that the Safety Plan is fit for the system being developed. The Project should be very clear as to what type of support it requires. The required ISA support will determine the nature and rigour of the ISA’s assessment but without this clarity the ISA remit may not be adequate to facilitate safety approval or conversely may provide in excess of what is actually required. Customers and Projects should be wary of remitting the ISA to give positive support for the proposed change which is being engineered by the Project, as this compromises the ISA’s role.

Projects should seek Safety Authority guidance on approval of their ISA remit. This will reduce the risk of the remit being inadequate to facilitate approval.

To ensure ISA support of the project safety approach and strategy and so reduce the risk of project re-engineering, an ISA remit should be issued during the first phase of the project’s life-cycle, allowing selection of an ISA in the same timeframe.

Customers and projects should expect ISAs to provide feedback on the remit. However the Customer and Project should ask for reasons for the feedback from the ISA. This should ensure that any refinement of the remit is justified and targeted to optimise the assessment.

4.2.2 ISA selection There are three main areas that Projects should examine when selecting a prospective ISA organisation:

Competency: Does the ISA have the ability to do the job? Projects should review and verify the prospective ISA’s competence accreditation and competence management arrangements for the particular Project. The Project/Customer should ensure that the accreditation is acceptable to the Safety Authority. The review should identify any gaps in the ISA’s competence, for instance the ISA might possess expertise in the required engineering areas but lack competence in railway operations.

Gaps in ISA competence are unacceptable. The ISA should demonstrate how all required competences will be provided and managed for the duration of the ISA’s employment on the Project.




The blend of skills required will vary according to the nature and needs of the Project. They could include: ISA duties; ESM principles; Quality Assurance (QA) processes; system, hardware and software component design; application design; discipline knowledge - railways as a whole, operations and engineering, renewals and maintenance; policies for normal, failure and emergency working and the transitions between these operating modes; the various railway standards; familiarity with the Grandfather Rights of old systems.

Projects should avoid over-reliance on any individual ISA engineer as this can introduce additional risk to the project. Therefore, projects should ensure that the ISA can offer alternative staff should the preferred individual be unavailable or, preferably, offer a team approach.

Independence: Can the ISA provide demonstrable independence? The Project should verify any prospective ISA declarations of independence. The project should ensure that the ISA organisation has had no involvement in the Project, nor is likely to have, that could be perceived as a conflict of interest. If this is not possible the ISA organisation may be able to manage it’s involvement to ensure adequate independence, for instance by clear delineation of involvement within the organisation.

See section 5.1.2 for further guidance on what to expect from an ISA in respect of independence.

Technical Assessment: Does the Independent Safety Assessor have a sound assessment strategy? The Project should review the ISA’s proposed technical assessment methodology, specifically, the extent to which the ISA can demonstrate a clear technical route to arriving at a final judgement. The approach should focus effort according to the known safety risks. The ISA’s assessment strategy should show:

• The plan for the work, identifying tasks and their dependencies.

• What evidence is to be collected and by which mechanism.

• An understanding of the ISA’s own assessment project risks and how they will be managed.

It is particularly important for the ISA to have a clear technical strategy for large and complex integration projects.

4.3 Working with an ISA

4.3.1 Roles and Responsibilities of the Project Team When working on a project that employs an ISA, the Project Manager’s responsibilities should include:

• Maintain the ISA Remit. The ISA remit should be regularly reviewed in order to ensure that it is still appropriate, otherwise the Independent Safety Assessment activities may not result in the anticipated level of support for the project. If a change to the project impacts on the ISA’s work then the remit should be updated accordingly. Remit updated should be agreed with the Safety Authority.




• Communicate Project Plans. In order that the ISA can perform activities in a timely fashion, without incurring delay or risk to the project, the Project Manager should keep the ISA aware of the project progress and plans. It is important that the ISA is informed of key project activities and meetings, so that the ISA has the opportunity to attend as an observer to understand the decisions being made.

• Co-ordinate Documentation. The Project Manager should provide the ISA with all necessary project documentation to an adequate standard and in accordance with the Project Plan, and ensure that Independent Safety Assessment documentation (reports and observations) is distributed appropriately within the Project organisation. It may also be useful to give early drafts of project documentation to the ISA for comment, as this can provide valuable feedback. Note, however, that reviewing early drafts of project documentation in the initial stages of the Project may not be appropriate, as the Project should not be unduly influenced by the views of the ISA while they are planning or developing their safety strategy.

• Manage the Independent Safety Assessor. The Project Manager should hold regular progress meetings with ISA to review their progress against plan, the associated risks, and to handle any issues that may arise. Project Managers should obtain status reports from the ISA so that in combination with progress meetings any risk of the ISA not meeting the remit is properly addressed in a timely manner. In general Project Managers should treat the ISA as any other critical dependency and manage them as such.

• Liaise with ISA. Project Managers and engineers should allow adequate time to be spent with the ISA to facilitate assessment activities and for participation in the ISA’s audits.

• Handle ISA Observations. The Yellow Book [1], section 14.6, states that Project Manager should respond to ISA observations. These responses should be provided in a timely manner to ensure that the Project is fully supported and not delayed. See also section 2.4.3.3.

4.3.2 Monitoring ISA Competence & Independence During the lifetime of a project, the ISA’s independence and competence needs to be monitored:

• Monitor ISA Independence. The Project should inform the ISA of any project changes that may impact on the ISA’s independence, for example the employment on the Project of a major new contractor. The Project should in these circumstances request a renewed declaration of independence. If instead the ISA declares a conflict of interest, then the Project, Safety Authority and the ISA should jointly address it.

• Monitor ISA Competence. If a change to the Project requires additional ISA competence, the Project should ensure that the ISA can provide the necessary competence – if not internally then through a suitably independent sub-contractor.




4.3.3 ISA Observations and Reports For projects to deliver within planned timescales, they should respond in a timely manner to ISA Observations and Reports. The Yellow Book [1] suggests specifying timescales for resolution of observations and ISA reports should provide suggested timescales. Any deviation from the timescales should be discussed with the ISA to minimise the impact in terms of delay and possible re-engineering.

It is quite likely that the Project will wish to debate or even challenge some ISA observations, and this is a perfectly reasonable way of resolving issues. The Project should respond to all observations, and should discuss contentious issues with the ISA in order to resolve them. In the case where the Project and the ISA cannot resolve observations, the involvement of the Safety Authority may be necessary to settle the matter.

All ISA observations and responses should be visible to the Safety Authority. The Safety Authority should be able to see a transparent audit trail of the Project’s interaction with their ISA.

4.3.4 Recognising the Boundaries Customers and projects should recognise the professional boundaries of their relationship with an ISA. Specifically, the following should be taken into account:

• ISA’s professional duty is to be independent. Customers and projects pay for the Independent Safety Assessment activity but should not direct or influence the ISA in any way which might compromise the ISA’s independence or the robustness of the assessment.

• ISA Guidance. Customers and projects often seek advice from their ISA, but this could breach independence. The ISA is professionally bound not to give advice to those projects which it is assessing. The ISA can only offer guidance where it is general and non-specific.

In this context, advice can be regarded as information that might influence the Project’s actions, whereas guidance is information which facilitates the Project’s own decision making. As an example, advice is the ISA assisting in formulating a safety argument whereas guidance is the ISA stating whether or not the safety argument is likely to be acceptable.



ISA Organisation Section 5

5 ISA ORGANISATION Organisations offering ISA services have professional obligations to ensure that the services they provide comply with regulatory and commercial requirements and are consistent with the ESM Independent Professional Review fundamental [1].


Design &Implementation

Installation &Handover

Operation &Maintenance

Decommissioning &Disposal


Demonstrate

Competence

Independence

Commercial Confidence

Manage

Competence

Independence

Commercial ConfidenceDocumentation records

Figure 3: Role and Responsibilities of an ISA Organisation (within EN 50126 Lifecycle)

Three particular issues which an ISA organisation should focus on are Competence, Independence and Commercial Confidence.

5.1 Demonstrate

5.1.1 Competence As an organisation, ISAs should provide a credible and independent means of demonstrating their competence. Competency and Accreditation schemes are readily available from professional bodies such as the Institution of Electrical Engineers (IEE) (sponsored by HSE) [2] and Network Rail [3]. Specific technical competences can be accredited externally, for instance the Institution of Railway Signalling Engineers (IRSE) licensing. Organisations should ensure that their accreditation is maintained, regularly reviewed, audited and is supported by a body of evidence. Should an organisation subcontract any Independent Safety Assessment activities, there is a responsibility to demonstrate that the subcontractor is competent for the activity assigned, in the same manner as the ISA’s own personnel. Appendix B provides a list of competence references.

The role of the ISA is to perform a single ESM activity – Independent Professional Review. Nevertheless, the Yellow Book Safety Responsibility, Competence and Training and Safety Culture Fundamentals are applicable to ISA organisations. An ISA organisation therefore needs to be able to demonstrate the adequacy of its safety regime in respect of roles and responsibilities, clearly indicating the internal safety approvals process and accountability.




ISA organisations also need to demonstrate a working Safety Culture. An appropriate Safety Management System alone is not adequate. The ISA organisation should be able to demonstrate, through the conduct of staff, a common understanding of why safety is embedded in every process. Good practice guides on how to demonstrate Safety Culture are available [5].

5.1.2 Independence For every Independent Safety Assessment activity tendered for or performed, the ISA organisation should be able to demonstrate the necessary level of independence. Yellow Book [1], Section 14.4.1 of Volume 2, gives recommendations on what level of independence should be adopted for any given system or equipment Safety Integrity Levels (SIL). Note: SILs are one way of specifying safety performance targets. To demonstrate independence adequately, at the start of a project an ISA organisation should:

• Check that the allotted safety performance target appears appropriate. This does not require a full assessment of the safety performance target derivation or apportionment, but rather a professional check based on the experience of the ISA. If a target has not been assigned, then the ISA should request that it is determined as soon as possible.

• Check that the level of independence being sought by the Project is commensurate with the safety performance target, and that the required independence can be provided.

5.1.3 Commercial Confidence Almost all commercial relationships are governed by Non-Disclosure Agreements (NDA), and consequently ISA organisations are bound to maintain the confidence of their customers, generally by law. However this highlights two conflicting needs:

• Maintenance of Customer competitive advantage; and

• Advancement of rail industry knowledge through lessons learnt.

ISA organisations are responsible for ensuring that the organisation and the teams and individuals within are bound by all NDAs which are in place.

Individual engineers who are Chartered Engineers are normally bound by their Institution’s Code of Conduct. For example, the IEE’s Code of Conduct states [4]:

Members shall not recklessly or maliciously injure or attempt to injure, whether directly or indirectly, the professional reputation, prospects or business of another.

However, ISA organisations should not seek to prevent their staff completely from using knowledge gained through one project on another project. Instead they should guide their staff on how to use commercially sensitive information.

This guidance should not compromise the ISA but make use of experience and knowledge where possible, for instance by generalising rather than making specific references to experiences and knowledge gained from previous projects. Any illustrative scenario should be used in a way that does not reveal the identity of the Customer, and key commercial information should always be removed.




The ISA has a responsibility to provide a report containing sufficient detail for the Safety Authority to make a competent decision and this cannot be compromised by commercial confidence issues. Consequently the ISA should ensure that a suitable agreement is in place, such as an NDA, between the Customer/Project and the recipient of an ISA report, prior to issue of the report.

5.2 Manage

5.2.1 Competence The ISA organisation should recognise that competence requirements may change over the project lifetime. The ISA organisation should monitor the competence requirements to ensure that changes to the Project which could impact on the ISA’s competence requirements are addressed as they arise. If new, or changed competence requirements cannot be met, the organisation has a responsibility to inform the Customer immediately.

The ISA organisation should be able to demonstrate systematic processes are in place for ensuring that competence is maintained and monitored throughout a project.

5.2.2 Independence After being appointed to assess a project, an ISA organisation should ensure on-going compliance against the Customer’s independence requirements. The requirements for independence may change over the project lifetime and the correct level of independence should be maintained. The ISA organisation should be able to demonstrate that systematic processes are in place, for ensuring that independence requirements are maintained and monitored throughout a project.

By monitoring independence, the ISA organisation should be able either to declare its independence or define the scope of any potential conflict of interest. For instance, a conflict of interest could arise when an organisation, which is already an ISA for a large project, undertakes some consultancy work for the same project. Where a potential conflict of interest has been identified, the project should be informed and alternative ISA arrangements made for undertaking the assessment in the identified area. The extent of the alternative arrangements can usually be limited to the area where independence cannot be achieved (depending upon the SIL of the requirements) – unless of course major conflicts have been found.

The ISA organisation should discuss major conflicts with the Customer and Safety Authority. If, after discussions, the ISA organisation can no longer meet the independence requirements then the ISA organisation should request to be relieved of its duty and contract, so allowing a suitable replacement ISA organisation to be found. In this case, the ISA should adopt a systematic handover process to ensure that the change of ISA does not introduce any safety risks.

5.2.3 Commercial Confidence An ISA organisation needs to monitor its performance regularly. The performance monitoring process should include checks to guard against breach of commercial confidence.




The ISA organisation should be able to demonstrate how commercial confidence is maintained and provide evidence of how staff are guided to use domain knowledge without breach of commercial confidence.

5.2.4 Documentation Control In principal, all documentation relevant to assessment work undertaken by an ISA organisation should be kept, in accordance with the Yellow Book [1] section 13.1.2, until there is no need to support further changes or accident investigation. This may necessitate retaining the documents until the change implemented by the Project is taken out of service.

The fact that the ISA organisation should retain documentary evidence as a matter of good practice does not obviate the need for other stakeholders to retain proper and complete documentation of the project (including the Independent Safety Assessment).



ISA Team and ISA Engineer Section 6

6 ISA TEAM AND ISA ENGINEER Depending on the size or complexity of a remit, an ISA organisation may appoint an individual or a team of assessors.







Raising major concerns

Final ISA Reports

Reporting

Handover

Required

Optional

DeliveringRisk-based Approach

Raising Observations

Methodologies, Tools & Techniques

Interface with NoBo

PlanningDevelop risk-basedapproachStaffing andCompetenceTasks and Timescales

Input to Remit

Establish NoBoInterface

Figure 4: Role and Responsibilities of an ISA Team or Engineer (within EN 50126 Lifecycle)

6.1 Planning Detailed plans are essential for safety assessment activities in the same way as they are for engineering projects. They provide a roadmap for successful completion of the assessment, setting safety assessment objectives from the outset which focus the resources on addressing the key safety risks.

6.1.1 Developing a Risk-Based Approach ISAs should undertake a risk assessment3 of the Project as the first step in the planning process. One way to do this would be to hold a brainstorm workshop with competent assessors. The results of the risk assessment should be recorded and used to formulate the technical assessment plan and to select the most effective types and levels of assessment activities. To perform the risk assessment the ISA team should consider the following:

• Risk Assessment Leader. Evidence of competent, strong facilitation skills with sufficient domain knowledge to direct and lead the session.

• Risk Assessment Process. Robust method, clearly focused on identifying the risks from all perspectives.

• Specific Competence and Domain Knowledge. Identify and record the assessors involved in the risk assessment, and their relevant skills and experience. This should be mapped to the characteristics of the system or equipment to show that all key safety areas have been considered.

3 The risk assessment should only be targeted at safety risks belonging to the Customer. These are risks which are safety aspects of the programme or product/system which if not adequately addressed could adversely affect the ISA’s support of the product or system. Some examples are unstable requirements or inadequate competency of key staff.




6.1.2 Responding to the ISA Remit Before embarking on the detailed planning of the assessment activities, the ISA should respond to the remit from the Project or Customer, identifying any deficiencies or excessive requirements in the remit, and providing substantiated, traceable reasoning for proposing any changes. The ISA should work with the Customer/Project to refine the remit, and the Project should liaise with the Safety Authority for approval if necessary.

6.1.3 Producing an Independent Safety Assessment Plan Assessment activities and techniques should be selected on the basis of the initial risk assessment (see section 6.1.1) in order to:

• collect evidence that an area presents an acceptable level of safety risk; or

• collect sufficient preliminary evidence on an area to decide what further in-depth assessment, if any, is required.

The technical assessment plan should indicate for each assessment task:

• The purpose of the activity, and specifically what evidence will be sought.

• Inputs, outputs and dependencies.

• The process to be used to undertake the activity, i.e. tools and techniques.

• The resource to be used, i.e. who will carry this out, and the timeframe for doing so.

• Interfaces for the task, and any additional inputs or outputs required for the task.

The technical assessment plan should ensure that emphasis is placed on technical assessment, supported by process assessment and audit. The plan should scope the assessment according to the remit. For example, the assessment of Operations and Maintenance manuals may or may not be required depending on the remit requirements.

The plan should also detail how observations from each assessment activity will be recorded, and to whom the feedback will be directed.

ISAs should plan for the regular review of project assumptions, dependencies and caveats4 in the assessment work.

All assessment activity planning should take into account the consequential workload placed upon the Project’s resources. ISAs should ensure that their assessment activities are planned so as to support the Project constructively and minimise the disruptive effect on the Project. For example, ISAs could liaise with other project auditors to combine visits.

The technical assessment plan should be regularly reviewed. The plan should be updated to address changes in the project as well as in response to interim assessment results. Updates should also take into account the ISA’s understanding of the project risks – this could affect the competences and tasks required for the remainder of the Independent Safety Assessment work.

4 An assumption is a statement that is taken to be true and relied on, even though evidence for its truth is not yet available. A dependency is an agreement between the project and another party that something will be in place before the implementation of the change. Caveats are conditions that people must respect after the change is put into operation for it to remain safe.




6.1.4 Staffing and Competencies An ISA team profile should be maintained as part of the technical assessment plan. The team profile should include:

• Team organisation chart; this should clearly show the line of safety responsibility, indicating which individual is the ISA Team Leader.

• Copies of all team member curriculum vitae (CVs), and records of their accredited competence.

• Competency Profile; cross reference of competences required for the Independent Safety Assessment activities against the competences of the team members.

• Training records, if any specific training has been undertaken specifically for the assessment.

The ISA team profile should include the same details for any subcontractors used by the team.

The ISA Team Leader is a key appointment. The ISA organisation should ensure that ISA Team Leader has all necessary qualifications as stated in the Yellow Book [1], section 14.4.2, and also that the Team Leader is acceptable to the Safety Authority and has the ability to lead a team of ISA Engineers.

ISA should adopt team working wherever possible. Team working offers a variety of benefits which ISAs should take advantage of:

• The diversity of viewpoints from a team approach can add significant confidence to assessment results.

• Reduced risk of the ISA not delivering to plan by avoiding over-reliance on an individual assessor.

• Reduced risk of assessors focusing on personal areas of interest at the expense of other issues.

• The opportunity to mentor less experienced staff and develop new assessors. Customers should be comfortable with less experienced staff given adequate levels of supervision.

On larger, complex projects often more than one ISA organisation is involved in assessing the change. In this situation, a single ISA organisation should be appointed as the Lead ISA to form the single point of contact for the Customer. The other ISA organisations involved then interact with the customer via the Lead ISA. Each ISA organisation, including the Lead ISA, will have its own ISA Team Leader.

6.1.5 Tasks and Timescales An Independent Safety Assessment project should be managed like any other engineering project to ensure completion to time and to budget, and therefore the ISA should produce an overall schedule, illustrating:

• when each activity will be carried out, and their dependencies;

• key milestones in the assessment;

• dates for the issue of Project and ISA deliverables; and

• date for completion of the assessment.




Plans should always include progress reporting and regular progress meetings with the Customer/Project.

Plans should ensure adequate provision for managing subcontractors.

6.1.6 Establish Notified Body Interface The use of a NoBo in projects on the European Railways was introduced in 2002, when the role and all those that interact with it were in a learning phase. This section of the Application Note is therefore subject to change as the role develops and matures.

Currently, the remit of a NoBo can be regarded as a super-set of the ISA remit. A NoBo will check conformance to standards and take input from an ISA to ensure that independent and professional judgement has been applied to the acceptability of safety risk. From this perspective a NoBo will be responsible for confirming that the system or equipment has undergone Independent Safety Assessment appropriate to the relevant essential safety requirements and notified national standards. The NoBo will look for ISA support of the system/equipment, and in some cases Safety Authority approval to satisfy the safety requirements component of the formal conformance assessment.

The ISA should be able to demonstrate systematic processes which facilitate efficient and cost-effective interaction with a NoBo, and that their activity will generate output and evidence sufficient for the NoBo’s needs as well as that of the Customer.

6.2 Delivering

6.2.1 Executing a Risk-Based Approach It is an ISA’s duty to ensure that the rigour of assessment is commensurate with the level of safety risk presented by the system or equipment. Using a technical assessment plan that is focused on Project safety risks should ensure the efficient use of the defined resources. Such an approach is also in accordance with the general good practice of adopting a risk-based mindset to safety engineering.

When the change under assessment is stated by the Project as being low risk, the ISA should not simply accept the assertion. Instead the ISA should check that the evidence to support the stated safety performance target is sound and consequentially ensure that the depth and rigour of safety assessment is adequate.

As one of the first assessment activities, ISAs should form a view on which standards are applicable to the Project. If the standards currently being adopted by the Project are different from those expected by the ISA, then the ISA should raise this issue immediately with the Project.

ISAs should review the Project’s specification, assessing it in context to ensure it is fit for purpose. The Application Note on Railway Level Issues provides guidance [1].




An ISA can add value to a Project by working with the Project to ensure key safety risks are addressed early. This can result in substantial cost and time savings. The ISA should not necessarily wait for final (approved) documents before commencing document reviews, so that safety risks are identified in the most cost-effective way. Confirmation should be obtained that the final (approved) documents do not differ in any material sense from the previously reviewed versions. If the final document is materially different, a further review should take place.

Occasionally, as part of a remit, assessors may be required to assess a system or piece of equipment that has already been accepted by another Safety Authority and so facilitate cross-acceptance – Appendix D provides guidance on this specific topic.

6.2.2 Raising Observations Wording of observations is important to ensure clarity and avoid mis-interpretation. Guidance is provided below to illustrate how the ISA can express observations objectively, whilst avoiding subjectivity and implied recommendations.

If the observation relates to an omission error then the ISA should use phrasing such as: “The section does not address … as required by … [standard]”.

Observations regarding the inadequacy of a safety argument are often more difficult to phrase than an omission. If the argument is unclear or inadequate then the ISA should justify why he/she considers this to be the case. For example “The argument that the risk associated with the human interface is ALARP is inadequate” could be phrased as “The argument that the risk associated with the human interface is ALARP cannot be justified as the ergonomic assessment has not yet been completed”.

When writing observations, assessors should avoid, or be careful in when using phrases such as the following:

• “The section does not appear to address …” Any observation raised should be apparent. If such observations are necessary then they should provide supporting explanation, detailing what is ambiguous.

• “The assessor is concerned that …” This implies subjectivity, observations should be objective. Observation of this nature can be helpful when the ISA is trying to ascertain if a problem is a symptom of a deeper underlying issue. However if this is the purpose of the observation it should be clearly explained, otherwise the phrase is best avoided.

• “The assessor would like to see assurance that …” This is subjective; however the observation could be objectively phrased as “The document should provide assurance that …”.

Observations should not use questions, particularly closed questions (i.e. requiring a “yes” or “no” answer), unless this is unavoidable. If questions are used then they should be open questions. For example “Is it possible to get within 1 metre where the EMC levels are high enough to be hazardous?” might elicit the response “No” which does not help the safety argument. It is preferable to phrase the observation “The document should provide assurance that it is not possible to …”.




Observations that are typographical errors (text, formula, reference, identifier) should simply list all such errors, preferably grouped together as a single observation. It is important that the focus of the assessment is on safety risk and whilst typographical errors should be recorded, their relatively (low) importance should be recognised. If a document is unreadable due to typographical errors or serious lack of clarity, then this of course should be raised as a more significant observation or in the worst case result in rejection of the document.

The classification of observations, such as A/1 (and their meanings), and how to aggregate observations, are detailed in section 14.6 of the Yellow Book [1]. However, ISAs should not limit themselves to raising issues which can be categorised according to the Yellow Book [1]. The ISA, during the course of the assessment, may uncover other issues which have significant impact on the project. The ISA should raise and record these issues even if they do not fall into any of the Yellow Book [1] categories and even if they are not directly safety-related. Nevertheless, the prime task of the ISA to identify safety-related issues, and Projects should not rely upon ISAs to identify issues which are not safety-related.

Important observations should be complemented by personal communication, by meetings or telephone conversations, in order for the ISA to provide early warnings to the project particularly for A/1 observations.

Records of discussions and meetings arising from ISA observations should be kept, in order to provide an audit trail of changes in the status of observations. This is important in order to avoid misunderstandings and to provide evidence to support the final status and ISA Report for the Safety Authority.

6.2.3 Raising Major Concerns If an ISA team member identifies a major safety concern he/she should discuss the concern at once with the ISA Team Leader. If the Team Leader agrees that this is indeed a major concern, then he or she should raise the issue immediately with the Project or Customer. The ISA Team Leader should not delay the information nor wait for the next report or formal reporting mechanism.

6.2.4 Methodologies, Tools and Techniques

6.2.4.1 ISA Methodologies

A wide range of methodologies are available to the ISA for assessment purposes. The ISA should choose those techniques that are the most effective for gathering the evidence an assessor requires. For example, if the competence of an individual needs to be assessed, then an interview combined with review of a CV will be much more effective than just a document review of a CV. However, the ISA should always be aware of the trade off between the importance of the evidence and the cost of collecting it using different methods.




There is a risk that ISAs rely solely on sequential document review as a means of assessing safety. Document reviews are not very creative activities and there is a danger of assessors being led through a train of argument rather than actively assessing it. In addition there are many safety issues which do not align with specific individual documents, but which emerge from a variety of project engineering activities, for example human factors, equipment timing behaviour and so on. Such issues are more effectively assessed as a topic. When undertaking a document review, assessors should create a checklist of what is expected so that the review identifies omissions as well as errors. Should the Safety Case documentation be inadequate then the ISA should not supplement or derive the Safety Case by undertaking alternative assessments.

Techniques such as audit, sampling, diverse analysis, witnessing, interviewing, traceability checks, vertical slices, inspection etc should be used in conjunction with document reviews to increase the confidence in the conclusions of the assessment. Brief descriptions of these techniques are provided in Appendix E.

The ISA should ensure that tools employed during the safety assessment are treated as configuration items, as stated in the Yellow Book [1], section 13.4.

6.2.4.2 Project Tools and Techniques

A safety assessment should review the tools and techniques used by the Project. The ISA should check that the project has selected tools and techniques that are fit for purpose, and that the Project has taken appropriate actions to mitigate any limitations associated with the use of the tools and techniques. For example if a design tool is only capable of checking certain software instructions, then the design should be subject to further checks. The rigour of the mitigating actions required should depend on the safety integrity of the equipment or system under assessment.

Further guidance is provided in “Validation of external items” Yellow Book [1], section 12.7.7.

Tools should be treated as configuration items as detailed in the Yellow Book [1], section 13.4. ISAs should therefore check that all relevant information is maintained for the tools in accordance with the Yellow Book [1].

6.2.5 Interface with NoBo The ISA should ensure that, if necessary, the remit makes adequate provision for interaction with the NoBo.

The ISA should facilitate NoBo requests for information and activities, in a timely and professional manner, where resource has been authorised for this purpose by the Project. The ISA organisation should be prepared to support the NoBo by providing access to documentation and personnel for interview. Should the NoBo, for any reason, be dissatisfied with the ISA, their work, independence or competence, the ISA organisation should be willing to accept further scrutiny, such as an audit.

The ISA organisation should ensure that there are systematic processes in place for interaction with the NoBo. However the ISA should only take instruction from the NoBo where authorised to do so by the Project/Customer or if contracted to do so directly by the NoBo.




6.3 Reporting

6.3.1 Final ISA Report The Yellow Book [1] provides skeleton report structures in Appendix B.4.

In addition, ISA reports should usually include:

• a clear statement of support or otherwise for the implementation of the change that the Project is engineering or where relevant for progression to the next project stage; and

• a prioritised list of recommendations, which should not only include a deadline for implementation but also state whether it is required prior to commissioning; and

• a finite validity period of ISA support, required in the situation where there are in-service issues yet to be resolved.

6.3.2 Documentation Handover ISAs should ensure that complete and comprehensive documentation records are retained for all assessment work. This includes records of all items assessed as well as the observations and reports generated during the assessment.

On completion of an assessment the ISA should ensure that the Project has in its possession copies of all the assessment reports and observation records, including the final report and any supporting letters or certificates, and that these are also available to the Safety Authority, where required.

6.3.3 Declaration of Completion The ISA organisation should not declare that an assessment is complete until the ISA is assured that all Safety Case mitigation measures can be implemented, assumptions recognised, dependencies assured throughout the life of the system (as required by EN 50126) and caveats agreed5. To achieve this, the assessor should seek assurance that the Project has appropriate processes in place to ensure implementation of all the mitigation measures and dependencies, and a mechanism for controlling compliance with caveats.

5An assumption is a statement that taken to be true and relied on, even though evidence for its truth is not yet available. A dependency is an agreement between the project and another party that something will be in place before the implementation of the change. Caveats are conditions that people must respect after the change is put into operation for it to remain safe.



Safety Authority Section 7

7 SAFETY AUTHORITY The Safety Authority is the person or organisation that accepts or approves a change for operational use.







Accepting a Change

Management of Supplier Commercial Confidence

ISA Deliverables

SA ISA FrameworkISA Accreditation

Accepting an ISA remitand appointment

Interaction with an ISA

Figure 5: Role and Responsibilities of a Safety Authority (within EN 50126 Lifecycle)

7.1 Safety Authority ISA Framework

7.1.1 ISA Accreditation As with all professions, it is important that safety assessment practitioners can be recognised as qualified and competent to carry out work. In order to assist Projects and Customers to select competent ISAs, Safety Authorities should provide or recommend mechanisms for accrediting ISAs. References to some mechanisms are provided in Appendix B.

Safety Authorities should periodically verify that the accreditation mechanisms are applied with adequate rigour and checks, for instance by periodic audits or submissions.

ISAs can be accredited at an organisational or individual level. ISAs can be accredited in the role of ISA for certain types of assessments or for a specific project or task. Safety Authorities should be clear as to the scope of accreditation of each ISA organisation.

7.1.2 Accepting an ISA remit and ISA Appointment When a project appoints an ISA, the Safety Authority should endorse or approve the appointment, assuring itself in particular that the ISA has the necessary competence. This should be done consistently, through a publicly stated mechanism. Some approaches to competence management have been listed in Appendix B.

Safety Authorities should approve or accept ISA remits, and should provide input to a remit when asked to do so by Projects (and in other circumstances where it is appropriate, such as high-risk changes).



Safety Authority Section 7

Safety Authorities should provide the Project with guidance, if requested, to assist in the selection of a competent ISA. If the Safety Authority does not approve of a selected ISA, they should clearly and immediately indicate this to the Project.

7.1.3 Interaction with ISAs A Safety Authority should expect an ISA to demonstrate their competence and independence throughout the Project lifecycle.

A Safety Authority should expect to be kept aware of the ISA’s view of the Project’s safety at all key milestones, and should expect the ISA to seek guidance when major concerns arise.

Safety Authorities should be prepared to provide guidance to ISAs by stating the criteria for accepting the Project’s change.

Safety Authorities should expect ISAs to be able to clearly articulate their view of the Project’s safety at any point in the lifecycle.

7.2 Accepting a Change

7.2.1 ISA Deliverables Safety Authorities should accept ISA deliverables only when they are clear in their conclusions and recommendations. Any ambiguity may conceal a safety risk or a lack of rigour in the assessment.

7.2.2 Management of Supplier Commercial Confidence To improve safety across the industry, Safety Authorities should provide means of communicating lessons learned from safety assessments, in order to improve safety assessment for all stakeholders – Customers, Projects and ISAs. For example standard risk registers for specific types of equipment or systems could be provided or maintained.

Safety Authorities should balance the need for safety improvement through lessons learned with protecting supplier competitive advantage. For example risk registers could provide details on the risk to be avoided without stating how individual suppliers have previously handled the issue.



Abbreviations Section 8

8 ABBREVIATIONS

ALARP As Low As Reasonably Practicable

CV Curriculum Vitae

DRACAS Data Recording and Corrective Action System

EMC Electro-Magnetic Compatibility

EN Euro Norm

ESM Engineering Safety Management

FAT Factory Acceptance Test

FMEA Failure Mode and Effects Analysis

GAMAB Globalement Au Moins Aussi Bon

HAZOPS HAZard and OPerability Study

HMRI Her Majesty’s Railway Inspectorate

HSE Health and Safety Executive

IEE Institution of Electrical Engineers

IPR Independent Professional Review

IRSE Institution of Railway Signal Engineers

ISA Independent Safety Assessor

NDA Non-Disclosure Agreement

NoBo Notified Body

QA Quality Assurance

SAT Site Acceptance Test

SHA System Hazard Analysis

SIL Safety Integrity Level

TSI Technical Specification for Interoperability



Acknowledgements Section 9

9 ACKNOWLEDGEMENTS This guidance was prepared with the help of the following people who provided their time and expertise as professionals committed to improving railway safety. Their views do not necessarily represent those of their employers. Their contribution is gratefully acknowledged.

Alex Stephenson, Alstom Transport Limited

David Jeffrey, Invensys – Westinghouse Rail Systems

Diane Whiteford, Aspect Assessment Limited

Francis How, Atkins Rail

Gab Parris, London Underground Limited

John Corrie, Mott MacDonald Limited

Ken King, Bombardier Transport

Peter Sheppard, Praxis Critical Systems Limited

A number of individuals and organisations have independently reviewed and provided input to the guidance, their contribution is also gratefully acknowledged.

Keith Williams, Aspect Assessment Limited

Ian Spalding, Aspect Assessment Limited

Keith Watson, Network Rail

IEE/BCS Independent Safety Assurance Working Group (IEE ISA WG)



Documents Referenced Section 10

10 DOCUMENTS REFERENCED 1 Engineering Safety Management Issue 3 Fundamentals and Guidance, Yellow Book 3,

RSSB, January 2000

2 Safety, Competence and Commitment: Competency Guidelines for Safety-Related System Practitioners © IEE 1999

3 Network Rail Corporate Independent Safety Assessor Accreditation, Crystal Blake, +44 (0) 20 7557 8513

4 Rules of Conduct, Institution of Electrical Engineers, Issue 3, 12th November 1996

5 Good Practice Guides: Assessing and developing the competence of senior management teams in strategic safety management, © Railway Safety, August 2001

6 Council Directive 2001/16/EC Interoperability of the trans-European conventional system, The Council of the European Union, 19th March 2001.

Council Directive 96/48/EC Interoperability of the European High Speed Rail System, The Council of the European Union, 23rd July 1996; and Council Directive 2001/16/EC Interoperability of the trans-European conventional system, The Council of the European Union, 19th March 2001.

1 Application Guidance Note, Railway Level Issues, Issue 1.0, Yellow Book 3

2 Proposed Cross Acceptance Processes for Railway Signalling Systems and Equipment, IRSE Technical Committee 6th Report, Final Draft, 13 Dec 2002

3 Sampling procedures for Inspection by attribute, BS6001:1-9 2002, British Standards Institute

4 Introduction to the ISO 2859 attribute sampling system, ISO 2859-0:1995, International Organisation for Standardization



Roles in each life-cycle phase Appendix A

APPENDIX A: ROLES IN EACH SYSTEM LIFECYCLE PHASE

ISA

Tea

mIS

A E

ngin

eer

ISA

Org

anis

atio

n

REQUIREMENTSDEFINITION

DESIGN &IMPLEMENTATION

INSTALLATION &HANDOVER

OPERATIONS &MAINTENANCE

DECOMMISSIONING & DISPOSAL

Cus

tom

erPr

ojec

tIS

AO

rgan

isat

ion

CONCEPT &FEASSIBILITY

Safe

tyAu

thor

ityIS

A T

eam

ISA

Eng

inee

r

Develop ISA RemitISA Selection

Seek approval of remit

Maintain RemitHandle ISA

ObservationsHandle ISA Reports

Liase with SafetyAuthority







Execute Trials

Close all open ISAObservations

Seek and obtain SafetyAuthority Approval

DemonstrateCompetence,

Independence andCommercial Confidence

Respond to remit,provide feedback

Monitor Competence,Independence and

Commercial ConfidenceInterface with NoBo





Monitor handover oncompletion

Technical AssessmentPlanning

Competent StaffingSchedule: Tasks &

Timescales

Use a risk-basedapproach

Raise observationsPerform Assessment

Audit Activities



Audit ActivitiesLiaise with Safety

Authority

Produce final ISA reportMake representation to

Safety AuthorityComplete & handover



Audit ActivitiesLiaise with Safety

AuthorityAssess trial results

Provide means ofrecognising or approving

ISAs as competentApprove ISA selectionRefine/Approve ISA

Remit

Review ISA outputManage Projects IPR

Mentor Project and ISA


Mentor Project and ISA


Mentor Project and ISAMonitor trials

Review Project SafetyCase and ISA

Assessment Report thenEndorse/AcceptProject's change

Disposal in accordancewith accepted provisions

of the safety case




This Appendix provides a system lifecycle view of the Independent Safety Assessment activities, to complement the guidance in the rest of the document and to provide some examples. The text that follows supports the table above and describes briefly what tasks the ISA would typically carry out at each lifecycle stage. This is designed to illustrate how the ISA’s role changes through the lifecycle, and how the ISA maintains independence.

The lifecycle and associated tasks follow those used in the Yellow Book [1], e.g. Table 11-2. Even though the description below allocates Independent Safety Assessment activities to particular lifecycle phases, this should only be used as indicative. The ISA may need to perform the activities in different phases from those shown, and/or perform more activities. The activities shown do not include any additional tasks that the ISA may consider necessary as part of a risk-based approach – see section 2.6.1.

A.1 Concept and Feasibility The Project will compile a remit and select an ISA (see section 2.4.2).

The ISA, Project and Safety Authority should work closely to set expectations at this stage, defining the evidence to be submitted for assessment, the appropriate assessment and audit programme, and agreeing how all parties will cooperate to ensure efficient acceptance.

The ISA would typically be involved at this initial stage to:

• assess the preliminary safety plan;

• assess the preliminary hazard identification including observing of Hazard and Operability Studies (HAZOPS);

• assess the hazard log management procedures, assess the establishment of the hazard log and audit that the procedures are being followed (the latter should continue throughout the lifecycle);

• assess all relevant project and development plans, including those for the acceptance process, in particular that suitable techniques and measures are proposed in order to meet the Safety Performance target (this may be deferred to a later stage if targets have yet to be determined);

• assess that the safety implications of the proposed approach have been analysed and the design is feasible to assure the acceptability of safety risk.

If the Project intends to use work conducted during this stage as the basis for options analysis later in the safety engineering lifecycle, then the ISA should assess the case for the selected option being ALARP.

The ISA should guard against making suggestions on how the Project should carry out its development except where it is obviously non-compliant with relevant standards or best practice.




If, when attending reviews or witnessing HAZOPS, the ISA believes that issues are not being addressed or hazards are being missed then these should be raised as this does not compromise independence. The assessor should ensure that they do not lead, manage or direct the HAZOP.

A.2 Requirements Definition The ISA would typically be involved during the Requirements Definition to:

• assess the hazard identification (again, observe HAZOP studies as necessary), hazard analyses, risk assessments and resultant safety requirements;

• conduct an independent review of the requirements (including interface requirements) to ensure that they are complete (checklists can be a useful aid for this task). This may also include audit of the Project against any requirements management plans and its use of any requirements management tools;

• assess the full Safety Plan;

• assess validation and acceptance plans, in particular that there are acceptance criteria for all safety requirements.

The ISA should ensure that they do not exert any influence over the development of the Safety Plan except, as before, where it is obviously non-compliant with relevant standards.

A.3 Design The ISA would typically be involved during the design to:

• assess the allocation of safety requirements to the design;

• assess any ongoing hazard analyses (e.g. System Hazard Analysis (SHA)) and risk assessments, in particular to check that the approach is ALARP;

• assess the design;

• perform safety audits of any design and associated verification processes against the plans.

During this phase key design decisions are being made, and is critical that the Project, not the ISA, owns these decisions. Therefore it is important that the working relationship between the ISA and the Project is kept relatively distant during this early stage to guard against the ISA unduly influencing project design decisions. After the initial draft and review of the design, the ISA can work more closely with the Project to ensure that the design will provide evidence to demonstrate that safety risk is acceptable.




The ISA should ensure that the entire safety argument is kept under sound configuration management. This means that all documents and evidence contributing to the safety argument should be kept under configuration control. Should the version of any of these items be changed (that is raised) the ISA should ensure that the project has fully understood the implications of the document or evidence revision. Consequently, the ISA should expect that the project has determined that the change is compatible with the existing safety argument or is taking steps to update the safety argument to ensure a consistent and compatible argument.

ISA should expect configuration management to extend to all Project documentation – plans, reports, safety cases, evidence, as well as relevant documents such Rule Books, Regulations and Zone Hazard Logs. The safety argument should be consistent in itself as well as with the operating policies of the railway on which it will be applied and maintained, and should be proportionate to the safety performance targets.

The Project should be aware that an ISA’s initial observations may arise due to unfamiliarity with the project. If the Project challenges or rejects ISA observations during this stage, it should provide sound and justified explanations why the observations are not warranted. This is to be expected and will provide important evidence of the ISA’s independence.

A.4 Implementation The ISA would typically be involved during the implementation to:

• assess the implementation, e.g. hardware manufacture, software code, unit test and verification, to ensure that it is in accordance with the design, and audit the implementation processes;

• audit the configuration management processes (audits may be carried out earlier in the lifecycle);

• assess any ongoing hazard analyses (e.g. Failure Mode and Effects Analysis (FMEA)) and risk assessments, in particular that the implementation is ALARP;

• assess any safety-related tools used by the Project;

• audit validation testing (e.g. Factory Acceptance Test (FAT)) particularly of safety requirements including witnessing of tests (perhaps on a sample basis);

• assess the safety arguments as evidenced in the Safety Case including compliance with the safety requirements.

Any audit samples should be selected in a manner than gives a valid representation or cross-section of the Project. The ISA should be wary of influencing the Project by announcing audit samples in advance.




During this phase the ISA is checking that the implementation matches the design, and it is therefore more of a verification audit type role. However, the Project should not expect the ISA to simply accept that the implementation matches the design. The ISA and the Project may need to work more closely during this phase, due to the nature of the assessment activities and this is reasonable, as there is less risk of the ISA unduly influencing design decisions.

A.5 Installation and Handover The ISA would typically be involved during the installation and handover to:

• assess that the system is installed correctly, typically by (sample) inspection, audit of installation procedures and assessment of installation records;

• audit acceptance testing (e.g. site acceptance tests (SAT)) and assess that the testers have appropriate independence and competence;

• assess that there are adequate fallback arrangements;

• assess that the relevant documentation has been transferred and that the receiving organisation has the appropriate competences and is adequately trained.

Following on from the previous phase, the ISA should focus on the Project’s assurance and monitoring activities, and should sample the handling of assumptions and dependencies.

For staged works and commissioning, the ISA should ensure that at each stage, mitigations are clearly stated, applicable, safe and time-bounded.

At handover, the ISA needs to ensure that all assumptions are met, that all dependencies are themselves assured, and that all caveats are fully accepted and resourced by the party accepting the responsibilities.During this phase the Project and ISA should expect to work together closely.

A.6 Operations and Maintenance, and Decommissioning and Disposal The implications of these phases should have been considered during the earlier phases. The ISA should, during the course of the assessment, ensure that there are adequate provisions for operations, maintenance, decommissioning and disposal of the change. Relevant Independent Safety Assessment activities may include:

• assessment of the operations and maintenance procedures where these are mitigations for particular hazards;

• ensuring that there is a suitable Data Recording and Corrective Action System (DRACAS) or similar system in place;

• ensuring that there are suitable procedures for controlling changes to the system and that these adequately consider safety (including appropriate regression testing);

• assessing training plans and competence requirements.



Competence Management Resources Appendix B

APPENDIX B: COMPETENCE MANAGEMENT RESOURCES The following list of references is not complete or exhaustive but should provide a useful starting point:

IEE Website: http://www.iee.org/Policy/Areas/SCC/index.cfm

Document: Safety, Competence and Commitment: Competency Guidelines for Safety-Related System Practitioners © IEE 1999

Rail Safety and Standards Board Website: http://www.rssb.co.uk

Document: Good Practice Guides: Assessing and developing the competence of senior management teams in strategic safety management

Her Majesty’s Railway Inspectorate (HMRI) Website: http://www.hse.gov.uk/press/2002/e02057.htm

Document: Developing and maintaining staff competence (HMRI Railway Safety Principles and Guidance Part 3 Section A) European Standards Document: EN 45000 Standard for Self Assessment of Quality Services



ISA Remit Checklist Appendix C

APPENDIX C: ISA REMIT CHECKLIST The body of the ISA Application Note highlights the ISA remit as a fundamental document to ensure that the ISA understand their responsibility and conversely that the Customer has clear expectations of the ISA. As a result, the following table has been provided as a structured checklist, or aide-memoire, for use during the production of ISA remits. The checklist is not exhaustive but rather is intended to be representative of the issues that a remit should include or address.

Introduction

Project Description

Project Context Programme Structure

Project Structure & Main Contact for the ISA

Approval Body and Approval process

Technical Scope

Risk Acceptability Criteria adopted

Independent Safety Assessment Context

ISA Structure. Indicate Lead ISA if more than one ISA organisation is involved.

Existing Work. Indicate any existing ISA work that should be relied upon, i.e. foreign ISA/acceptance, or prior assessment reports.

Requirements of the ISA

ISA Technical Scope. Indicating interfaces with other ISAs if applicable

Independence (Request external accreditation if appropriate)

Competence Infrastructure

(Request project Rolling Stock

examples if appropriate) Operations

Signalling Command & Control

Telecommunications

Energy

Maintenance

Electro-Magnetic Compatibility (EMC)

Human Factors

Safety Culture

Domain knowledge (i.e. Railway system where change is being applied)

Standards to be applied

Level of support required



ISA Remit Checklist Appendix C

Plan of Required ISA Activities

Outline activities

Deliverables

Timescales, Programme Inter-dependencies & Milestones

APPENDIX D: CROSS ACCEPTANCE

D.1 Definition Cross-acceptance is a generic term for the processes involved in approving equipment (and systems) where the approval for use is largely based on the previous use and acceptance of that equipment by another railway authority.

Under the European Interoperability directives [6], if an item being cross-accepted is an interoperable constituent with a valid EC declaration of conformance then it is illegal to subject that constituent to any further safety tests. This does not, of course, preclude the Customer assessing it against their own business requirements.

D.2 Background Cross-acceptance provides a number of challenges to an assessor:

• It is relatively new to the railway industry and thus the processes in use by railway authorities are still maturing.

• There is no common approach to the process of cross-acceptance although some railway authorities have produced guidance6. Other bodies have also produced useful reference documents, see reference [2].

• Dependent on the age of the system to be cross-accepted, the level, depth and quality of the safety information available to support acceptance can be extremely variable.

• The approach to safety approval/acceptance varies between countries and railway authorities (eg preferences for qualitative analysis/quantitative analysis etc) and the approach to acceptability of risk is variable (eg Minimum Endogenous Mortality, ALARP, Globalement Au Moins Aussi Bon (GAMAB)).

All the above means that an assessor should make certain that the assessment approach is closely tailored to the equipment under acceptance and the previous approval approach used.

6 Acceptance Service Guidelines: Guidance for cross-acceptance of products onto Network Rail controlled infrastructure, Network Rail Plc, 15 June 2000



Cross Acceptance Appendix D

Note, only when the CENELEC standard PrEN50129 is in wide use across Europe and safety cases take on a standard form can cross acceptance become a more “standardised” process. It should be noted that PrEN50129 omits to address “Grandfather Rights”, and any change involving cross-acceptance would have to take this into account on a case by case basis.

D.3 Stages in Cross-Acceptance When a cross-acceptance approach is being used for approval, the assessor should consider the following steps (based on the guidance for cross-acceptance previously referenced):

• The ISA should review the definition of the system as applied in its native environment against the operation and application of the system in its target application environment.

• The ISA should review the product acceptance of the product in its native environment, including its scope and the competence of the certification body. The ISA should check that all assumptions, dependencies and caveats are clearly identified.

• The ISA should review the product acceptance process that was applied for the product in its native context, to ensure that it is equivalent and consistent the acceptance process that is normally applied by the railway authority in the target application environment. In particular, the ISA should check whether there has been any independent assessment or review by a third party, the level of independence, the approach taken and the competences used.

• The ISA should review the history performance in order to ascertain if the products have achieved the specified performance levels in service in the native application (in particular reliability and DRACAS, or equivalent, information).

• The ISA should check that the project has adequately defined and described how the product will be applied in the target environment and that the differences arising between the application in the native and the target application have been clearly identified. For example differences may include the philosophy regarding operation or maintenance between the countries. In particular, the ISA should review the safety activities that have been carried out to identify differences (for example, hazard identification concentrating on the differences, derivation of safety requirements, risk assessment, production and maintenance of the hazard log). The ISA should pay particular concern to whether any changes are required (in either hardware, software, application or procedures) in the introduction of the equipment to the new application and how this affects the risk. If the changes are significant then the ISA should consider whether this undermines the cross acceptance argument.

• The ISA should review the resultant definition of the safety requirements for the target application.

• The ISA should review the evidence for the product’s compliance with relevant standards and check that any derogations are clearly identified.



Cross Acceptance Appendix D

• The ISA should review that the project has presented satisfactory and robust arguments for the safety of the new application and that the relevant safety requirements and criteria have been met.



APPENDIX E: EXAMPLES OF ISA METHODOLOGIES This section provides a brief overview of some methods which can be used to supplement the basic ISA methods of document review and audit. It is not meant to provide a definitive or exhaustive guide.

Sampling – The objective of sampling is to enable the ISA to gain an appropriate level of confidence in a specific area, when assessing all the related evidence is not practical, or indeed an effective use of ISA resource. The need for sampling can often arise in the case of well-defined engineering processes which generate large volumes of evidence or documentation, such as FMEAs or software module designs. For example, in the case of an FMEA, the ISA may sample how particular subsystems have been analysed, or sample how particular failure modes have been assessed across the whole system. The objective and rationale behind any sampling should be defined and documented. Should the sampling reveal significant problems then further assessment would be required.

There are two standards which are available and are useful references if using sampling as a technique [2,4].

Vertical Slice Analysis – The objective of vertical slice analysis is to trace the mitigation of a hazard throughout the system life-cycle. The ISA would assess the safety requirements derived from a particular hazard, the design requirements derived from it, specifications, implementation and then the supporting verification and validation evidence. Thus the assessor builds an overall picture of how the safety argument for a particular hazard was developed throughout the life-cycle. Vertical slice analysis gives a quite different viewpoint to the majority of assessment activities that tend to focus on a particular life-cycle phase. An ISA could use such an approach when assessing a hazard of particular concern, or when assessing the overall effectiveness of a Project’s systems engineering process.

Interview – The objective of an interview is to gain insight into issues such as safety culture, competence, or safety concerns regarding engineering aspects of the project. Output from an interview should be used to guide the assessor to assess or audit specific areas to collect objective evidence in order to substantiate, or not, views expressed during an interview. The remit, objectives and output of interviews need to be agreed carefully beforehand.

For example, the ISA may be concerned about the safety culture on a project and may conduct a series of structured interviews to collect evidence to substantiate, or allay, this concern.

Diverse Analysis – The objective of Diverse Analysis is to gain confidence in some critical aspect of a safety case by performing an analysis that differs in some way from that performed by the Project. This does not mean completely repeating the analysis, but performing enough to gain confidence in the analysis under review.

For example, the ISA Team could perform an independent HAZOP on an area of particular concern and compare the results with the Project. Given the criticality of the hazard identification process, this could well be justified, and may provide significantly more confidence than an individual assessor reviewing the Project’s HAZOP report.


Page 43 of 43

Published in May 2003 by: Rail Safety and Standards Board Evergreen House 160 Euston Road London NW1 2DX Phone: +44 (0)20 7904 7777 www.rssb.co.uk Distributed by: Praxis Critical Systems Limited 20 Manvers Street Bath BA1 1PX. Phone: +44 (0)1225 466991 www.praxis-cs.co.uk Copyright © RSSB 2003 You can download further copies from: www.yellowbook-rail.org.uk
