Using an LLM to analyze a PCBA

Perhaps it is now time to start sharing my real work. As a start I am sharing the results of a test I performed using an LLM to analyze a PCBA.

The objective of this test was to explore using an LLM to analyze a schematic and set of fabrication drawings.

Conclusion

My overall feeling matches a statement generated by Claude. Always great to get an extra set of eyes reviewing.

Everything that was generated by the LLM required human engineering review, but in general the results were highly accurate and useful. When asked to perform analyses with insufficient information, and I didn’t let the LLM not perform the action, the amount of speculation (hallucinations) were generally understandable and easily dealt with.

Generic LLMs were used for the analyses. No custom LLMs or finetuning was performed.

Generally results were good. On one of my major objectives, generate a BOM from a legacy schematic only available as a PDF, was a complete failure.

Background

1. Schematics and fabrication drawings used were open sourced by Tesla Tesla documents
2. Anthropic Claude and OpenAi’s ChatGPT were used as LLMs

PCBA Design packages

Three legacy (12+ years old) professional (Tesla Roadster) PCBA design packages were analyzed.

1. Vehicle display system (VBS) - For most of the VBS tests only the schematic was provided to the LLM. package
2. Battery monitoring system (BMS) - For the BMS tests the schematic and fabrication drawings were provided. package
3. Heating, ventilation, and air conditioning system (HVAC) - Only schematics are available. package

LLM used

Primarily the chat browser interface to Claude 2.1 was used. If the LLM being used was not identified then that was being used.

Summary of analyses conducted

1. Provide a general review of the design.
   • Evaluate the design tradespace of the design
2. Evaluate the engineering team. Who did the design? Where was it done?
   • Identify when the PCBA was designed
   • Identify where the PCBA was designed
3. Estimate physical characteristics of the PCBA
   • Estimate the size of the PCBA
   • Estimate the power of the PCBA
4. Suggest design improvements to the design
   • What design changes would you recommend to reduce cost
   • Modernize the design (the designs I analyzed were a dozen+ years old)
5. Generate a BOM
   • Compare the schematic with the pick and place report
6. Evaluate product release
   • What was the likely design test process/procedure
7. Evaluate manufacturing
   • Manufacturing volume
   • Estimate the manufacturing cost of the PCBA. Parts, assembly, test, etc.
   • What is the likely manufacturing test process/
8. Evaluate Part selection
   • Microcontroller
     • Evaluate the selected microcontroller used
     • What microcontroller would you suggest using in a redesign
   • LCD display
     • Evaluate the selected LCD display used
     • What LCD display would you suggest using in a redesign
9. Reliablity

Defects and challenges during the analyses

1. Failed to generate a useful BOM from a schematic only PDF
2. Some analyses generated hallucinations or overly speculative results
3. Claude complained about performing a number of actions.
4. Once you identify to Claude that it is hallucinating it becomes less willing to speculate.
5. The chat interface wasn’t always the best UI.
6. The lack of file generation made some operations challenging.
7. Some responses were long and required repeated “continue” requests to obtain the complete
8. Claude needed to be asked questions for it to understand the design. Until I asked about VDS U2 Claude hadn’t realized that the design included a touch screen.

While Claude and ChatGPT failed to generate a useful BOM from a schematic only PDF I do have a method that works. Perhaps I will write about it in a later blog post.