BN-G-UP-203 Guide for QA Evaluation of Suppliers

1.

QA Program/System

a) Has the supplier established and documented a formal system of quality management?
b) Has the supplier documented his system for inspection?
c) Do the supplier's plans for inspection provide for all specified requirements to be met?
d) Are the supplier's arrangements for inspection to be performed out-side his premises acceptable?
e) Are the supplier's inspection records capable of providing objective evidence that the specified require­ments have been met?
f) Has the supplier a system for final inspection and test?
g) Are criteria for acceptance or rejection clearly docu­mented

2.

Organization

a) Has a representative responsible for inspection been appointed?
b) Has the representative the necessary authority, re­sponsibility and ability to perform his functions effec­tively?
c) Has the supplier identified, and assigned responsibility for, the functions and activities directly affecting quality?
d) Has a "Management Representative" been appointed?
e) Has this representative the necessary authority, responsibility and ability to perform his functions effectively?
f) When applicable, is the Purchaser's Representative afforded reasonable access and facilities? g) Has a representative responsible for contacts with Authorities been appointed?

3.

Design Control

a) Does the supplier's system provide for a planned program of design and development?
b) Does this system cover the aspects of reliability, maintainability, safety, design review, value engineer­ing, standardization, interchangeability and documenta­tion control?
c) Are design and development responsibilities clearly assigned?
d) Are design reviews carried out as part of the suppli­er's system of design control?
e) Does design review provide for assurance that the technical data created by design reflect contract requirements?
f) Is there an adequate system for controlling the issue and recall of this data?
g) Is there a procedure for proposing, approving and implementing design changes?
h) Is there appropriate control of design changes requiring approval by the purchaser?
I) Do the supplier's control procedures include his sub­contractors, and does the supplier monitor subcon­tractor changes requiring his approval?
j) Does the supplier's system provide adequate technical data for transition to the production phase?

4.

Procurement Document Control

4.1

Inspection and Test Procedures

a) Does the supplier have appropriate documented in­spection procedures for, as a minimum, those inspection stages where inspection is performed to demonstrate conformance to specified requirements?
b) Are such inspection procedures clear, current and complete?

c) Do they include criteria for acceptance or rejection and identify the inspection equipment to be used?

4.2

Technical Data and Changes

a) Does the supplier have procedures that ensure that only the latest applicable technical data are used for inspection purposes?
b) Is there an acceptable system for controlling changes to technical data or instructions arising therefrom?

5

Written Work Instructions/Workmanship

5.1

a) Are documented instructions available for work operations where lack of instructions would adversely affect work performances?
b) Are such work instructions clear and complete?
c) Do they establish acceptable quality standards for the work operations covered?
d) Are they compatible with associated inspection and testing?
e) Is proper use made of the work instructions?
f) Are the work instructions systematically reviewed?

5.2

Workmanship

a) Has the supplier established satisfactory standards of workmanship?
b) Do these provide an objective basis for decisions on acceptability of material?
c) Are only qualified personnel used for work requiring special skills or training?

6.

Documentation and Change Control

a) Does the system provide for clear and precise stipu­lation of responsibilities in documentation issue and change control?
b) Are changes made in writing?
c) Is the system of recording changes satisfactory?
d) Are obsolete documents promptly removed from all points of issue or use?
e) Are changes related to subcontracted material approved by the supplier or, under special arrangements, by the subcontractor?
f) Are the consequences of changes notified by subcon­tractors evaluated by the supplier?
g) Are the consequences of changes on interchangeability and spare parts made known to the purchaser?

7.

Control of Purchased Material/ Purchaser Supplied Items and Services

7.1

Purchasing

a) Does the system ensure that material and services supplied by subcontractors meet contract requirements?
b) Does the system provide for the selection of subcon­tractors on the basis of their quality capability?
c) Does the supplier review his subcontractors' perfor­mance at intervals consistent with the complexity and quality of the product?
d) Do supplier records provide evidence that the suppli­er's controls, and those of his subcontractors, are adequate to assure the quality of purchased material and services?

7.2

Purchasing Data

a) Does the system ensure that material and services supplied by subcontractors meet contract requirements?
b) Does the system provide for the selection of subcon­tractors on the basis of their quality capability?
c) Does the supplier review his subcontractors' perfor­mance at intervals consistent with the complexity and quality of the product?
d) Do supplier records provide evidence that the suppli­er's controls, and those of his subcontractors, are adequate to assure the quality of purchased material and services?

7.3

Receiving Inspection

a) Does the supplier inspect incoming material to the extent necessary upon receipt?
b) Does the supplier adjust the extent of receiving inspection on the basis of objective data?
c) Does the supplier ensure that material conforms to the applicable physical, chemical and other technical requirements, using laboratory analyses as necessary?
d) Is tested, approved material identified and carefully segregated from that not tested or approved?
e) Does the supplier have effective controls for preventing the use of non-conforming incoming material?
f) Are there adequate procedures for providing subcon tractors with appropriate data regarding unsatisfactory quality?
g) Has the supplier adequate controls for ensuring correction of subcontracted non-conforming material?
h) Does the supplier's system provide for the immediate recall and replacement of material released for production prior to completion of receiving inspection and subsequently found to be non-conforming?

7.4

Verification of Purchased Material

a) Do subcontracts afford the Purchaser's Representative the right to verify conformance at source when required by the contract?
b) Are copies of documents necessary for such verification made available to the Purchaser's Representative in good time?

7.5

Purchaser Supplied Material

a) Does the supplier examine "supplied material" upon receipt for damage, quantity, completeness and type?
b) Are there precautions and inspections during storage against damage and deterioration and to check on storage life limitations?
c) Is all "supplied material" properly identified and protected from unauthorized use or improper disposal?
d) Have procedures been established for notification to the purchaser of any loss, damage, malfunction or deterioration of "supplied material"?

8.

Identification and Control of Materials

Included in number 7

9.

Control of Special Processes

9.1

Special Processes

a) Do any of the supplier's manufacturing processes fall into the category of special processes?
b) Does the quality system ensure that appropriate and detailed work, inspection and test instructions are provided and used for any highly specialized or complex processes?
c) Is equipment used in special processes suitable?
d) Are personnel performing work on special processes adequately trained and qualified?

9.2

In-Process Inspection

a) Is inspection performed during manufacture on all characteristics which cannot be inspected at a later stage?
b) Are the inspection procedures carried out, test devices used, records kept, etc., suitable?
c) Do the operations performed by the supplier meet any special process requirements specified in the contract?
d) Does the supplier periodically monitor these operations to ensure their continuing compliance with requirements?

10.

Inspection/Manufacturing Control/Sampling Procedures/ Inspection Status

10.1

Inspection Representative

a) Is there a nominated person responsible for the inspection function?
b) Does this person have sufficient authority to ensure the level of quality required?
c) Are his inspection responsibilities compatible with other responsibilities he may have?
d) Has he adequate facilities to carry out his duties?

10.2

Manufacturing Control - General

a) Are all production processes accomplished under controlled conditions?
b) Does control include necessary documented work instructions, adequate production equipment and appropriate working environments?
c) Do necessary work instructions provide criteria for determining whether production, processing and fabrication work is acceptable or unacceptable?
d) Does the quality system provide for monitoring both the issue of necessary work instructions and compliance with them?
e) Are physical examinations, measurements or tests of materials provided for each work operation where appropriate?
f) When direct inspection is not practicable, does the system provide for indirect control by monitoring of processes?
g) Are both physical inspection and process monitoring used when either alone would be inadequate, or when required by the contract?
h) Is inspection and process monitoring accomplished systematically and are records kept?
i) Are unsuitable inspection or monitoring methods corrected promptly?
j) Is conformance to documented inspection methods complete and continuous and are corrective measures taken when non-compliance occurs?
k) Are acceptance and rejection criteria provided for all inspections and monitoring actions?
l) Is accepted and rejected material properly identified?
m) Does the quality system ensure provision of the proper processing equipment as well as the necessary degree of certification, inspection, authorization and monitoring for specified and complex processes?
n) Do established standards of workmanship and acceptance and rejection criteria provide an objective basis for decisions on acceptability?
o) Can work be released without the agreement of the authority responsible?

10.3

Indication of Inspection Status

a) Does the supplier employ an effective system for indicating the inspection status of material?
b) Is the inspection status of material readily apparent?
c) Is batch or lot identity maintained throughout the manufacturing process where necessary?
d) Is non-conforming material identified as such?
e) Is the method of identification satisfactory?

10.4

Sampling Procedures

a) Are recognized standards on sampling being utilized or are supplier-designated sampling plans available for review by the Purchaser's Representative?
b) Do supplier-developed sampling plans provide valid confidence and quality levels?
c) Does the supplier enforce all of the conditions required for the application of the sampling plans used?
d) Are proper records maintained of sampling procedures and results?
e) Is there a clear identification of characteristics to which sampling is applied?
f) Does the supplier meet the conditions necessary to ensure effective sampling, e.g. batch sizing, randomness in selection, homogeneity order ranking, 100% inspection of rejected batches or going to a more stringent inspection level?

10.5

Alternative Inspection Procedures and Equipment

a) Is the supplier's proposal to use alternative inspection procedures and equipment backed by valid evidence?
b) Do supplier proposals provide information on how the continuing suitability of alternatives will be maintained?

11.

Test Control (Later)

12.

Control of Inspection Measuring and Test / Instruments/ Equipment

a) Does the supplier have a documented measurement and calibration system?
b) Are necessary gauges, testing and measuring equipment available and used?
c) Is the accuracy of the inspection and checking devices provided by the supplier known?
d) Has the supplier an acceptable method of maintaining this level of precision?
e) Are results of calibration checks known?
f) Is calibration performed at established intervals?
g) Are records of calibration checks available?

Note: For further typical questions reference is made to Appendix 3

13.

Material Handling, Storage, Shipping and Preservation

13.1

General

a) Does the system provide for the identification, as necessary, of the material from the time of receipt until the supplier's responsibility ceases?
b) Are adequate work and inspection instructions prepared and implemented for the handling, storage and delivery of material?
c) Are handling, storage and delivery procedures and methods monitored as part of the quality system review?

13.2

Material Handling

a) Has the supplier instructions or procedures, where necessary, to control handling and transport operations?
b) Are special crates, boxes, containers, trucks or other transportation vehicles provided for handling material?
c) Are handling devices periodically inspected for cleanliness and suitability for use?
d) Is material suitably protected when passing through or held in areas that may contain harmful contaminants?

13.3

Storage

a) Are there procedures and regular schedules for the inspection of products in storage and are these procedures adequate to prevent deterioration or damage?
b) Is there a procedure to ensure that items that can corrode or otherwise deteriorate during manufacture or interim storage are properly cleaned and preserved?
c) Are all required critical environments maintained during storage?

13.4

Delivery

a) Is all material to be stored or shipped properly identified and labelled?
b) Are all shipments prepared and transported in conformance to specified requirements and applicable carrier regulations?

14.

Completed Item Inspection and Testing

a) Are completed items given a final inspection and test to establish overall quality?
b) Does the final testing meet the requirements of the relevant specification?
c) Are inspection and test problems or deficiencies promptly reported to the appropriate authority? d) Is there re-inspection and re-test of all items that are re-worked, repaired or modified after initial and product testing?

15.

Non-conforming Materials/Items

a) Does the supplier have an effective system for controlling non-conforming material?
b) Does the supplier properly identify, segregate and dispose of non-conforming material?
c) Are the procedures for repair and re-work of non-conforming material documented and acceptable?
d) Is relevant scrap and re-work data maintained and available for review?
e) Do repair and re-work activities comply with documented procedures?
f) Is repair work carried out in an acceptable manner to meet the original specification?
g) Is repaired work re-inspected to original specification(s) (or quality levels)?
h) Are holding areas adequate for the segregation and storage of non-conforming material?
i) Has the supplier nominated personnel with authority to be responsible for review and designation of non-conforming material?

16.

Corrective Actions

a) Does the system provide for prompt detection of inferior quality and for correction of its assignable causes?
b) Is adequate action taken to correct the causes of defects in material, facilities and functions, e.g. design, purchasing, testing?
c) Are analyses made to identify trends towards material non-conformance?
d) Is corrective action taken to arrest unfavorable trends before non-conformances occur?
e) Does corrective action extend to subcontractor material?
f) Is corrective action taken in response to user data?
g) Are data analysis and material examination conducted on scrap or re-work to determine extent and causes of defects?
h) Is the effectiveness of corrective action reviewed and subsequently monitored?

17.

QA Records

a) Are there records of essential quality assurance activities?
b) Are there records of essential inspection and test activities?
c) Are there effective means for assuring the currency, completeness and accuracy of records?
d) Do inspection records contain all essential data?
e) Do they show the results of inspections performed with regard to the appropriate acceptance criteria? f) Do they show that corrective action is taken when necessary?
g) Are records analyzed and used for the purpose of management action?
h) Are there satisfactory arrangements for the storage and retrieval of records?

18.

Audits/Review

a) Does the supplier's management carry out periodic reviews?
b) Do these reviews make use of the findings of internal audits?
c) Does the supplier have a documented procedure for the audit of the system?
d) Are procedures for audit documented?
e) Are written audit procedures sufficiently comprehensive to provide objective evidence of the system's effectiveness?
f) Are audits planned and conducted systematically?
g) Is effective corrective action taken when revealed as necessary by the audit findings?
h) Are records of audits maintained as part of the supplier's records?

A.

Planning

a) Has the supplier obtained all the information needed to execute the contract?
b) Has the supplier conducted a complete review of his contract to identify and provide for special or unusual contract requirements?
c) Has the supplier initiated quality planning prior to starting work?
d) Does the supplier's quality planning include identification of the need for developing new testing and inspection techniques?
e) Does quality planning provide for identifying material characteristics, or new or unique manufacturing processes, that affect end product quality?
f) Are the plans and practical equipment compatible, for example, tool precision and inspection and measuring instruments?

B.

Training

a) Are arrangements for personnel training satisfactory?
b) Are training records maintained?

C.

Subcontractors

a) refer to 3.i;
b) refer to 6.e and 6.f;
c) refer to 7.1.a thru d;
d) refer to 7.3.f and g;
e) refer to 7.4.a.

1.

Calibration System

a) Are there prescribed procedures for the control and calibration of all measurement standards and measuring equipment?
b) Is it effective and complied with?
c) Are the appropriate management responsibilities documented?
d) Does the system provide for the prompt detection of deficiencies to prevent subsequent inaccuracies?
e) Is there a procedure to ensure corrective action?
f) Does the supplier have properly defined and appropriate limits of calibration uncertainty?

2.

Periodic Review of the Calibration System

a) Is the procedure for the review of the measurement and calibration system documented?
b) Does it cover all facets of the system and is it effective and complied with?
c) Are the reviews conducted systematically and is the periodicity satisfactory?
d) Are records of the reviews maintained and do they provide objective evidence of the effectiveness of the system?
e) Is senior management informed of the results of the review and is corrective action taken?

3.

Planning

a) Are the needs of calibration and measurement properly planned before starting new work?
b) Are the necessary measurement standards and measuring equipment determined?
c) Has the availability of all required equipment been determined?
d) Are the calibration and measurement requirements reviewed to determine special, unusual or "state of the art" needs?
e) Is there an appropriate area allocated for carrying out all in-house calibration work?
f) Has the level of skill and training required by the calibration and measurement personnel been determined?
g) Are controlled environments provided where necessary?
h) Are there any measurement requirements that cannot be satisfied that are relevant to the products? i) Are reports of corrective action used to update planning schedules?
j) Have the particular needs of automatic test equipment (ATE) been allowed for, and what plans exist for the verification of software?

4.

Measurement Limits

a) Does the calibration system identify the source and magnitude of uncertainties associated with calibration and product characteristic measurements?
b) In specifying product characteristic limits, are errors of significance taken into account as appropriate?
c) Does the system identify the particular techniques used for the determination of errors and uncertainties of measurement?

5.

Documented Calibration Procedures

a) Are there prescribed procedures for controlling the calibration of measurement standards and is measuring equipment available and used for product verification?
b) Where there are no in-house procedures, are appropriate and identifiable published standard practices or manufacturer's written instructions prescribed?
c) Is there a clearly identified procedure for each item of measuring equipment?
d) Is the latest issue of procedures identified and available?
e) Is a check carried out to ensure that written procedures are adhered to?
f) Where a subcontractor is used in the measurement chain and special measurement procedures are required of him, are they adequately prescribed?
g) Are procedures prescribed for the control and/or measurement of special environmental requirements, if appropriate?

6.

Records

a) Is there a system of record keeping that is appropriate to the calibration and measurement system in use?
b) Are all the records required by Appendix 4 (when applicable) maintained?
c) Do the records, as necessary, include details of calibration controls, environmental data, designated error limits and information necessary to establish traceability?
d) Is simple and rugged measuring equipment included in the records?
e) Do the records indicate that the equipment is capable of performing measurements within the designated limits?
f) Does the record system allow for calling forward, at the appropriate interval, equipment requiring calibration?
g) Does the record system indicate that equipment which is only calibrated in part?
h) Does the system include the retention of calibration certificates or data used in support of all calibration of measuring equipment?
i) Do the records document the software used for automatic calibration and test equipment procedure?
j) Is the change procedure, relating to calibration and measurement, documented?

7.

Calibration Labelling

a) Is there a prescribed system of labelling or coding that identifies the calibration status of measurement standards and measuring equipment?
b) Is equipment that is not fully calibrated or has limited use fully identified?
c) Is there a prescribed system to ensure compliance with requirements where labelling is not practicable?

8.

Sealing for Integrity

a) Are there adequate procedures for sealing against misuse or accidental shift of present controls?
b) Have any adjustable devices on measurement standards and measuring equipment that are not necessary to the normal operation of the equipment been fully sealed or safeguarded at the time of calibration?
c) Are the methods and materials used for sealing such that any tampering will be noticeable?
d) Is the sealing such as to prevent access into the equipment?

9.

Intervals of Calibration

a) Have calibration intervals been established for all measurement standards and measuring equipment?
b) Have the calibration intervals been established from the equipment manufacturer's recommendations or knowledge of equipment stability, purpose, degree of usage, calibration records and experience?
c) Are calibration intervals adjusted on the basis of trend data obtained from previous calibration records?
d) Are calibration intervals adjusted on the basis of the usage of the equipment?

10.

Invalidation of Calibration

a) Do the prescribed procedures ensure the immediate removal from use, or conspicuous identification of any measurement standard or measuring equipment that:
(1) has not been calibrated in accordance with the established time scale?
(2) has failed in operation in any measurement parameter?
(3) shows evidence of physical damage?
(4) is suspect for any reason?
b) Are the procedures effective and complied with?
c) Do the procedures provide for immediate notification of equipment failures or damage likely to have compromised product quality?
d) What procedures or embargoes are in operation to prevent the unofficial use of privately owned equipment that is not covered by the calibration system?
e) What safeguards are prescribed to ensure that equipment is not brought into use without adequate checking and calibration?

11.

Subcontractors

a) Do procedures ensure that a subcontractor employs a measurement and calibration system that complies with the requirements of this appendix?
b) Is the responsibility accepted by the supplier for ensuring that the procedures employed by a subcontractor for calibration and measurement work are suitable and are properly documented?
c) Is there a system for continuously evaluating subcontractors?

12.

Storage and Handling

a) Is there an adequate system for the handling, transporting and storing of measurement standards and measuring equipment?
b) Is the system under the control of the calibration department?
c) Do the procedures include a requirement for reporting damage, abuse or deterioration?
d) Are storage and handling arrangements periodically reviewed, particularly taken into account the increasing sophisticated nature of the equipment?

13.

Traceability

a) Can all calibrations performed in-house or by subcontractors be traced through an unbroken chain of properly conducted calibrations to a national or international measurement standard?
b) Are calibrations involving the use of natural physical constants or ratio type self-calibration techniques adequately controlled?

14.

Cumulative Effect of Errors

a) Do the procedures take into account the cumulative effect of errors and uncertainties in carrying out calibrations?
b) Is suitable corrective action taken when the total uncertainty could compromise calibration or measurement capability?
c) Is there a procedure for determining the various errors and uncertainties of equipment or a system? d) Is the total calibration or measurement uncertainty given on all certificates of calibration?

15.

Environmental Control

a) Is environmental control provided where necessary?
b) Are the environmental conditions provided appropriate to the uncertainty level of the calibration and measurement work undertaken?
c) Are controlled environment areas properly laid out, monitored and controlled?
d) Are environmental monitoring devices properly maintained and calibrated?
e) Are environmental compensating corrections applied to calibration and measurement data when necessary?
f) Do the procedures for calibration and measurement of the equipment indicate the required environment?

16.

Evaluation of Calibration System

a) Is reasonable access made available for the evaluation of the system?
b) Are adequate facilities provided?

17.

Training

a) Have all personnel performing calibration functions the appropriate experience or training?
b) Is the experience or training applicable to the type of calibration work undertaken?
c) Are calibration personnel sent on training courses?
d) Is the competence of staff reviewed at the planning stage