Scope Document

• Deliverables What is the actual outcome? An executable-file?, new HW with new SW? old HW with new SW etc.
• Definition of Done Is the R&D project also responsible for starting up Production? Market introduction? Training of sales-org?
• Platform Do we expect to run a desktop program on Linux, Windows or Mac? If it's embedded - do we already know that it will run on Linux, or something else? If it's an app - is it IOS or Android - or both? If it's a cloud-solution - are we already "married to" Google, Amazon or Microsoft? - or do we spend time on evaluating each of these? Will we phase the introduction on some of the above - e.g. Windows first, then Mac?
• Geography Is it worldwide, regional or local? This can mean a lot in relation to compliance - and thereby the work that goes into fulfilling this.
• Staffing Will the project run on existing resources? If so, someone needs to decide what to do less of. If not - then who?

Requirements Documentation

• Testable
• Traceable (where does it come from) If your company loses a big order, or decides to pull out of a market, it is important to know which requirements are not needed anymore. If you decide to involve companies to help test the prototype, it makes sense to go with the ones that will see most of their requirements fulfilled. If you are ever in doubt about a requirement, you need to be able to find its origination and dig into it.
• Prioritized Use e.g. "classes" - like "A" for mandatory, "B" for important, "C" for Nice-to-have and "Z" for not-needed. It is a huge advantage also to know what is NOT required.
• Atomic It is bad practice to put several requirements into one. If a device needs to be operational between -20 and +70 Degrees Celcius, you should create a requirement for each of the two limits. In the end you may be forced to change e.g. the lower limit to -10 degrees. No need to go through everything that builds on the upper limit.

Design

• Presentations/Discussions and Documentation: UML Diagrams
  • Use-case-Diagram
  • Sequence-Diagram
  • Class-Diagram - Containment,(Inheritance)
  • Structure Diagrams
  • Deployment Diagram
  • Statemachine-Diagram
• Parallel work: Scheduler/Kernel/RTOS/OS in IoT book and Microcontrollers
  • Building Blocks: Tasks & Interrupts, Messages, Queues, Shared Memory
  • Parallel Processing: Reentrancy, Thread-Safety, Atomic operations, Mutexes, Critical sections, Priority (-inversion), Round-Robin
• Breaking down work: Gang-of-4 Patterns
  • Layers
  • Creational: Singleton, Factory, Prototype, Builder
  • Structural: Façade, Proxy, Decorator, Composite, Adapter, Bridge
  • Behavioral: Observer (Publish/Subscribe), Iterator, State, Strategy, Chain of responsibility, Command, Interpreter, Mediator, Memento, Template, Visitor
• Low-level
  • Data-Structures, Memory-layout, Memory-types, Compiler & Linker
• Database: Entity-Relations, Physical/Logical, Integrity, Constraints
• UX/UI: Real-World Match, Status Visibility, Consistency, Standards and more (this is guidelines more than a design view)

Coding

It is important not to choose the latest business fad as your main embedded language. The code is probably going to live long, and you will need developers who understand the language in the future. Languages such as C and C++ are still very popular in embedded, and Python is really strong in testing as well as application layer in larger embedded systems. See also the Trends page.

• A basic code-guideline
• Static Code Analysis (preferably as automated checkin-gates). The free CPPCheck is an excellent beginning.
• Minimal formal review - like Pull-Requests (often related to git workflows). It is very safe to sit at each our own desk and do Pull-Requests, however, there is a lot of value of doing this together. This allows everyone to hear the answers to questions and thus improves the shared knowledge.
  • Formal Inspections is even better

I discuss Static Code Analysis as well as Secure Coding Guidelines here: Cyber Security.

Debugging

• Low-level Single-stepping, breakpoints, tracing, memory-dumps etc
• Medium-level Print statements that are always in the code - normally disabled, but can be run-time enabled within "domains" like "network", "audio", "SPI" etc and typically also more or less verbose. Typically enabling all these debugging printouts at once will drag a system down, but you should have the extra CPU-power to open up for selected parts.
• High-Level Statistics - e.g. on mailboxes; how much queue, which messages in the queues? etc.

When we talk about debugging, most people think about single-stepping and breakpoints. This is great if you have an idea of where in your code to look. To get to this point you may need to start with an eagles view - hence the other debugging levels.

Unit and Integration Tests

When is it a unit-test and when is it an integration test?

The important thing is that the team performs tests on their code before they release it to the main branch (often called master). And that even more testing is used when blocks are integrated. In many cases you can use the same tests for both types.

Obviously the new code should be tested. I recommend that small "test-journals" are written. It is so little work to copy a screen dump from an IDE or a terminal to a Word-doc, but it can save you so much time later, when you start doubting what really worked and what you did or did not test. A test-journal is not a formal document as such, but do yourself a favor and keep it for at least a year.

Don't get caught up in religious wars on unit tests. Some believe that everything should be unit-tested, but there are many cases where your time may be better used in other kinds of tests. James Coplien has some data-based ideas of why unit-tests can be waste.

"Smoke-tests" of the existing code can also be advantageous. The more these can be automated - the bigger the chance of them being run.

Regression Tests

The more people are involved in a project - the more important it is that nightly regression-tests are run. These will often catch unforeseen side-effects. If the project is e.g. PC-software it is possible to run many parallel tests, and therefore have a great code-coverage in the tests.

Embedded projects may depend on hard-to-get HW-prototypes and may have sensitive wiring, or expensive prototypes. Such setups are not easily parallelized. This means that in order to run many tests, each needs to be of short duration. In this case it may make sense to run the longer-running tests over weekends.

The faster a bug is found after being introduced - the faster it is to fix it.

Typically regression tests are run on the main branch. By testing nightly we get a chance to find a defect within 24 hours. But a defect may have disturbed many people just by entering the main branch. For this reason many teams have gated check-ins where fast tests are performed before the code enters the main-branch.

Another approach is to have a "semi-main" branch where code is checked in, and tested nightly. Only if this is passed is the code allowed to find its way into the main branch.

When regressions tests are not just pass/fail, but also qualitatively measure parameters like Idle Time (%), Power-usage (W), Dynamic Range (dB), Packet-loss (%), GPS-deviation (m) etc - then you can depict these measurements on the Y-axis on a graph with time (or builds) as the X-axis. This becomes a valuable trend-analysis, where "jumps" are often related to check-ins - on purpose or not.

This builds up to CI - Continuous Integration.

Other Tests

Production Tests

Many embedded software teams are taken by surprise when the company's own production department requests tests. Such unexpected work can be very time-consuming. It makes a lot of sense to raise the question early:

Customer Premises test

A power-up self-test is often necessary, but more elaborate tests can also be relevant. This depends on your product. An example is expensive B2B products that may be hard to return for service and are hard to live without while at service. The producer and the customer has a shared interest in uncovering as much as possible on premises. It is often possible to ask the customer to run specific tests and return the results in mails.

In the case of consumer products, it is more and more common for the product to autogenerate logs that are transmitted to a cloud-server when convenient - if the customer has approved it.

Defects

• Symptoms What is observed. NOT what do you think is wrong (put it in e.g. details)
• How to recreate
• Details
• Who reported the defect
• What characterizes the system Versions of SW and HW.
• Who is responsible for the defect now
• State Travels between e.g. New, Don't fix, Works as designed, Fixed in code, Released and possibly more states.

Version Control

• You have a version control system
• Check-ins are "atomic" and related to a single story/feature or defect. This allows you to rollback if needed, with known consequences.
• For a given release you know what changes it contains ("shopping-list")

Compliance

Before the project is too far it is important to understand the compliance requirements, so that the relevant work can be planned into the project.

• Usage-Domain Medical and Automotive are known for their many compliance requirements, but e.g. Sound-Level meters and other devices used for measuring also have their domain-specific compliance requirements.
• Implementation-Domain If a product has a wireless connection, there are many compliance requirements. These are country specific - like the FCC-rules in United States. Approval in one EU-country is however, valid for all.
• General If you create HW for the EU you need CE-marking. This covers a lot of specific measurements like e.g. EMC and ESD. There are also IP-ratings (Ingress-Protection) for dust and fluid penetration. These are generally not required, but is something you can certify a product against - and then use in your marketing (business-to-business).

Steering Committee Meetings

Most companies have Steering Committees that help the projects to stay on track. It is important to understand that often the same people are in many SCMs. To ease their work, and their ability to compare projects,  all projects should report in the same "template" - often Power-Point. This will contain slides on:

• Status on Milestones Some of the best milestones are demos. Most people can understand a well presented demo, and it gives credibility to the other measures.
• Other Progress Indicators A good friend used to show a block-diagram of the product with components and interfaces marked with Green, Yellow or Red symbols for their current status. This is very relevant when introducing new HW with embedded SW.
• Risk Analysis The classic format is a table with a line per risk. In the line we have the textual description, an estimate on probability, an estimate on effects on the product launch if the "thing" happens and some kind of multiplication of the latter two. A risk could have "High" probability of happening (marked red), but a "Low" effect (marked green) ending as a yellow result.
• ETA Estimated Time of Arrival. Typically based on a curve of e.g. "Story Points" left over time. This is called a "Burndown Chart". When will the graph cross the time-axis?
• Hours Spent If a project is going nowhere it could be because the developers are doing everything else. Keeping some kind of track of the effort going into a project makes a lot of sense. You may be surprised.
• Percentage of regression-tests passed and code-coverage in regression tests All tests may be passing, but what if we only test 10% of the code? There should be a goal for code-coverage - typically 100% is not really doable.
• Market fit Has the market changed since the project was started? Has competitors introduced "nasty" products? Is the overall economy dictating new habits? Has the product moved?