SLS Flight Software Safety Issues at MSFC (Update)

[bananas]

http://nasawatch.com/archives/2016/09/sls-flight-soft.html

SLS Flight Software Safety Issues at MSFC (Update)
By Keith Cowing on September 26, 2016 5:48 PM. 17 Comments

Keith's 26 September note: Shortly after this original posting on 9 September Andrew (Andy) Gamble was summoned to MSFC Center Director Todd May's office to talk about QD34 issues. During subsequent closed door meetings MSFC management decided that they did not have any software problems - when in fact they do. A few days later Steven Pearson, Deputy Director of NASA MSFC's Safety & Mission Assurance Directorate, suddenly announced his retirement after 37.5 years at NASA. This is not surprising since the internal assumption is that the first SLS flight will have a built-in risk of failure of around 8%. This risk is being "baked in" to the design of SLS in part due to decisions being made at MSFC about software and avionics - decisions that are being made so as to not surface troublesome issues that no one wants to deal with. One can imagine that safety folks at MSFC are nervous.

Meanwhile, contractor employees working for QD34 who have surfaced the issues I reported earlier have suddenly found their funding yanked. Moreover, employees who are leaving or thinking of leaving as a result of raising concerns - are being specifically blackballed - by name - by MSFC management. Potential employers are being told by NASA MSFC that they risk wining new contracts - or losing existing contracts - if they hire these individuals.

<snip>

This is no way to build a rocket folks.

Keith's 9 September note: NASA's SLS program has been experiencing budgetary and scheduling issues for years as noted in multiple GAO and OIG reports. The program has also had problems with technical issus such as software and avionics. Multiple sources report that one of the places experiencing significant problems is QD34 at NASA MSFC - where significant SLS flight software safety issues are mounting - issues that no one else is hearing about. The Branch Chief for QD34 is Andrew Gamble. NASA MSFC management - and perhaps the NASA OIG might - want to pay that organization a visit.

Keith's 9 September update: According to an internal memo there is a "lack of understanding by QD34 of the intent of NPR 7150.2B (A or B) [NASA Software Engineering Requirements] and of CMMI [Capability Maturity Model Integration] when it comes to a higher level of process, procedure, etc. rigor expected for Class A software than for Class C" and that actions by NASA MSFC QD34 management represent "a direct attempt by our QD34 customers to intentionally minimalize the differences to avoid making matters complicated and having to make any corresponding changes for SLSP at this time."

<snip>

Some of the comments with the article:

CaringInAction • 16 days ago

I work for QD34 as a contractor. I would like to remain anonymous for fear of retribution from the branch chief and I would like to keep my job and others have been fired. I have seen QD34 management manipulate data and discourage people from speaking out using intimidation. NASA upper management needs to confront this problem or it will keep getting worse.
[hr]
RocketScientist327 • 2 hours ago

You are not alone... which is why websites like this are so critical. It is not that we will change anything however we can document, document, document so when the wheels do come off the wagon we can say see I told you so.

I do not have any faith and confidence in OIG anymore. They are well aware of the problems here.
[hr]
Resident4 • 2 hours ago

I agree, OIG is a puppet organization on so many levels. I've seen them ignore outright fraud many times.

[Blue_Tires]

1. How do you "bake in" a risk of failure?

[MicaelS]

2. Shuttle O-Rings comes to mind. n/t

[GoDawgs]

3. It's a blind assumption that a problem will be solved in a future design iteration

But it's either too late, too expensive, awaiting finalization of another component/milestone, or not important enough to worry about now. While its common especially for the first flight prototype, it's dangerous to let these types of problems sit around & fester because they creep into the accepted design.

[Blue_Tires]

5. ok I get it now...

[GoDawgs]

4. Is the Inspector General office the only oversight entity the engineers/contractors can

Appeal to?
Commenter Rocketscientist327 is expressing a feeling I've seen several times internally on teams and even among subs for larger projects in the engineering world.