The algorithm to control the insulin pump can be described well using the tools for the software engineering discipline for the specification of "real-time" systems and embedded controls. This will fit the artificial pancreas algorithm project naturally if we remember that our expectation for closing the loop between the insulin pump and CGMS will sometimes be very helpful, but at other times will not. We must learn to recognize the natural modes of the technology and then design the algorithm to control the artificial pancreas in each of the natural observed modes. For example, the closed loop control system may be great for controlling the basal rate of the insulin pump in some modes, but in others the CGMS feedback is better used for learning the C/I ratio and the insulin sensitivity factor. Yet in other modes such as when a bolus is delivered and a meal eaten, then a predictive approach is best until the carbs eaten and insulin-on-board are in balance and the loop can be closed again.
States and modes are important in software design when used to control the activation and deactivation of the software components of a complex algorithm. Different software processes will each control the insulin pump in its different natural modes (deliver basal rate only, deliver bolus, stopping all delivery, etc.), but those in turn will be controlled by another "executive" software process. The executive activates and deactivates the real-time processes according to state-machine software which mimics the natural observed modes of the insulin pump and CGMS. Once you can design a detailed state machine that captures the sequences of state changes and the logic for changing states, then you are well on your way to being able to control the insulin pump with simple processes that sum up to a powerful algorithm. The greater benefit however is that the discipline invested in a state-based control scheme will pay off in a robust design -- one whose response to failure modes is predictable and repeatable.
The states and modes of the artificial pancreas algorithm[]
The artificial pancreas algorithm will be controlled by a software state machine which mimics the natural states of the predictive insulin pump algorithm with the loop closed by the CGMS data. When possible (in multiple modes), the algorithm will close the loop with real-time blood glucose data to control the basal insulin delivery and achieve a desired blood glucose level. In one of the closed loop modes, the CGMS feedback will additionally be used to learn the basal rate for that time of day, but in another closed loop mode the CGMS feedback will be used to learn the C/I ratio for a particular time of day (e.g., breakfast). In other modes, closed-loop control is not possible, and yet in other modes all insulin delivery must be stopped. We can envision all of the different modes for our artificial pancreas algorithm. These modes, named descriptively according to their function, are as follows:
Control Basal Insulin Closed-Loop Mode[]
This is the mode we first thought the closed loop artificial pancreas could be in all the time. However now we know this concept only works well when we are not eating and the sensors are working great. In this mode, the insulin pump and CGMS will work together to close the loop between measured blood glucose and the insulin delivered from the insulin pump. Higher and/or increasing BG values will cause immediate increased insulin delivery, whereas lower BG values will cause a corresponding reduction. A generic PID controller would work fairly well in this mode only, and is a reasonable starting point of the algorithm in this mode until better data is collected. Also, in the closed loop mode we will be simultaneously using adaptive variables to "learn" the basal rates for the pump user as a function of the time of day. When BG feedback shows that the basal rate actually needed right now does not match the predicted basal rate, then the closed loop approach will fix both the BG level error right now and also "nudge" the basal rate for this time of day for future use.
Control Basal Insulin Open-Loop Mode[]
This is the mode we already have (an insulin pump is delivering insulin according to a predictive algorithm which was designed by our experts), but it will be better now to think of it as "waiting to close the loop again as soon as we can automatically do so." If diagnostic software detects that CGMS data is suspect, then the algorithm will operate in this mode and wait to go back to the closed loop mode.
Emergency Halt Mode[]
In the event of pending or onset of severe hypoglycemia, the pump in this mode will be commanded to halt all delivery and alarms will be issued. This mode should be seldom or never used. The pump will have to be manually restarted again after triggering the halt mode.
Deliver Correction Bolus Mode[]
In this mode, a correction bolus will be delivered, and then we will wait to close the loop again. CGMS feedback will not be used to control the basal rate.
Evaluate Correction Bolus Mode[]
In this mode, CGMS feedback will be used both to learn the insulin sensetivity factor (ISF) for the correction bolus as a function of the time of day, as well as to control the basal rate to adjust the correction bolus if the error is small. The basal rate will NOT be learned in this mode.
Deliver Prebolus Mode[]
We already know that you cannot close the loop on the basal rate when the blood sugar is spiking as a result of eating a meal. We also know that the insulin pump and CGMS seem to work better together if we "strike the spike" by prebolusing insulin 15 minutes before eating. The algorithm in this state delivers a manual bolus set up by the user and then waits for the next state for the opportunity to close the loop again. Two distinct states are needed: in this one we do not want to adapt the C/I ratio or adjust the basal in any way, but in the next state we want to do exactly that.
Evaluate Prebolus Mode[]
In this mode we can close the loop on the insulin-to carb ratios for prebolusing and adapt that variable, as well as evaluate the total insulin delivery from the bolus and adjust it (i.e. an automatic correction) as BG trends show it to be inappropriate. In this mode, the basal rate from the pump will be considered as part of the total insulin delivery with the bolus. The basal rate can changed for a short duration to provide the bolus correction. Control of the basal rate in this mode will close the loop with CGMS data and control the total insulin delivery to achieve the insulin balance and the target BG values before returning to closed loop basal control. We do not want to adapt the basal rate values however, as we consider the variance more likely to be the result of an error in carb estimating or in the carb-to-insulin ratio rather than an error in the basal rate.
Deliver Missed Bolus Mode[]
In this mode (having observed the very undesirable trend in blood sugar for a large rapid rise that indicates a missed bolus), we can close the loop and fix the gross BG error while being careful NOT to adapt any of the basal rates or insulin-to-carb ratios of a pre-bolus. The missed bolus is a fact of life in the world of insulin pumps that are operated manually, and compensating for them should have a beneficial impact to HbA1c values. That said, there are at least two approaches to automation of the insulin pump: the automatic mode where no buttons need to be pushed, and the semi-automatic mode where buttons are pushed to start a control action but then that action is completed automatically. Relative to our artificial pancreas algorithm, you could say that the prebolus mode is the semi-automatic because the pump user enters grams about to be eaten, pre-boluses 15 minutes before eating, and then the pump automatically continues and completes the bolus goes back to closed loop mode as soon as it can. On the other hand this missed bolus mode is more like the fully automatic mode: the algorithm would sense that the blood sugar is spiking (where the rate and magnitude are both sufficient to indicate a missed bolus) and then the algorithm automatically delivers the insulin bolus without user input. Sometimes also referred to as an adaptive bolus, this missed bolus will deliver what is effectively a combination bolus consisting of a spike bolus together with an extended bolus. The spike bolus will be proportional to the spike in the blood sugar, and the level of the extended bolus will be proportional to the rate of change of the falling blood sugar after the bolus (i.e. the derivative function of the PID), and finally the duration of the extended bolus will be adjusted by the algorithm in response to the blood sugar level relative to the total insulin-on-board until an insulin balance is observed (i.e. the integration mode of the PID). It appears rather trivial at this point that with the 15 minute delay in the delivery of insulin subcutaneously, this adaptive bolus (while very cool!) will never perform as well as the pre-bolus in terms of avoiding the high blood sugar spike and the hours of exposure to hyperglycemia that follows the spike, and so for now we will consider this adaptive bolus as the best approach only for a missed bolus. In this mode, the adaptive bolus is started. The spike bolus is delivered, and then the extended bolus is continued.
Evaluate Missed Bolus Mode[]
In this mode, the adaptive bolus is continued with the insulin tail working while the basal rate is controlled by CGMS feedback. In this mode, the basal rate will not be learned however.
While all of the states and transitions are important, they are not all typical. When everything is working well, then there is little need for the algorithm to stray from the path of the primary transitions shown in black. When not bolusing, the artificial pancreas is controlling the basal rate with CGMS feedback and learning the basal rate. When prebolusing, the artificial pancreas predicts the insulin needed and then waits for insulin on board to be reduced, doing little more than waiting and watching. Then the artificial pancreas will transition again and evaluate the meal bolus, controlling the basal rate to balance the insulin with the insulin on board and the CGMS data, while learning the correct C/I ratio for that meal to continually improve the next prediction. When done, the artificial pancreas transitions back to the first mode and controls the basal rate to balance CGMS data while learning the basal rate. As this continues throughout the day, the artificial pancreas algorithm will learn the basal rate for different times of the day and will learn C\I ratios for different times of the day. Unless the user forgets to bolus, or the CGMS sensor faults, or there is a large insulin imbalance caused by a highly unusual event, then the pump will stay in primary path.
State transitions[]
State transitions are events that cause us to change states. Every state has a functional mission to perform, and so we need state transitions to know when we need to change states because the mission of the existing state has been accomplished, or else because a higher priority mission in another state has been identified. The control strategy for the artificial pancreas algorithm depends equally upon the state transition criterion together with the state missions themselves.
The transitions among the states shown in the state machine figure above are as follows:
T1: Severe hypo detected[]
(i.e. transition from numerous modes to the "Emergency halt" mode)
The purpose of this transition is to recognize severe hypoglycemia from low blood sugar trends, and then act to stop the insulin pump from delivering any more insulin. If moderate hypoglycemia is detected, then the closed loop should reduce the insulin delivery to compensate. If that is not effective in worst cases, then halting the total delivery is necessary.
The logic of this transition is if the blood sugar is less than 55 counts for a short duration, or is less than 65 counts for an extended period, then we want to transition to the halt state.
T2: CGMS sensor fault[]
(i.e. transition to "Control insulin pump open loop" mode)
The purpose of this transition is to open the control loop when software diagnostics detect that the CGMS data is suspect. CGMS sensors have a limited life and need to be replaced. The lifetime of any single sensor will vary, and the care of the user of the CGMS will also vary before replacing the sensor. It is necessary both to stop adapting the basal rate variables when the CGMS data is suspect, as well as stop controlling the basal rate itself. The transition takes us to open loop mode, where the insulin pmp will continue with a fixed programmed basal rate.
The logic of this transition is: If CGMS data = min or max, or if CGMS data /= BG test +/- 30 counts, or if CGMS diagnostics = fault, then the next state is "Control basal insulin open loop."
T3: CGMS Data Valid[]
(i.e. automatic transition from "Control basal insulin closed loop" mode to "Deliver Correction Bolus" mode)
The purpose of this transition is to leave the open-loop mode and go to the closed-loop mode when the CGMS sensors have been replaced. We will want to start the closed loop mode with BG in range (otherwise the algorithm would "learn" incorrectly) so an automatic correction bolus would be given. When possible, closed loop control of the basal rate will begin in the "Monitor correction bolus state", and when BG is close to a target range then the algorithm will transition to the closed-loop mode and learn the basal rate.
The logic of this transition is: If the present BG is greater than the target value of 110 counts plus 70 counts, then the AP state changes to "Deliver Correction Bolus" state.
T4: Correction bolus needed[]
(i.e. a transition from the closed loop mode due to an automatic correction bolus, or any mode when a manual correction bolus is given).
The purpose of this transition is to leave the closed loop or adaptive modes and thereby stop learning the pump parameters when suspect. The high BG values which necessitates the correction bolus would only cause the basal rates to adapt falsely upward.
The logic of this transition is: If a correction bolus is started manually, or if the present BG is greater than the target value plus 70 counts, then the AP mode changes to "Deliver Correction Bolus."
T5: Manual bolus 15 minutes before a meal[]
(i.e. typically a transition from "Control basal insulin closed loop" mode to "Deliver Prebolus" mode, but it could actually transition from any state when a manual bolus is given)
The purpose of this transition is to recognize that we need to leave the closed loop modes where we are learning the pump parameters, and wait to close the loop again.
The logic of the transition is: a manual bolus was started.
T6: Missed Bolus identified by a BG spike[]
(i.e. transition only from "Control basal insulin closed loop" mode to "Deliver Missed Bolus" mode)
The purpose of this transition is to recognize that we need to leave the closed loop mode where we are learning the basal rates of the user, and go to the Missed Bolus" mode where we can still close the loop on BG but we don't want to learn a new basal rate that will be wrong. We can give a good missed meal bolus but do not want the algorithm to learn from it.
The logic of the transition is: we were in the closed-loop mode when a large rapid rise in blood sugar was detected.
T7: After bolus delivered and IOB<50%[]
(i.e. transition from any "deliver bolus" mode to any "evaluate bolus" mode)
The purpose of this transition is to start adapting the pump parameters. The 50% IOB rule is a bit of a guess -- but in general by the time you are half-way through a bolus, then you have a pretty good opportunity to look at the insulin on board and the blood sugar level and start closing the loop to the pump.
The logic of this transition is: IOB<50%
T8: Bolus Complete (IOB<5%)[]
(i.e. transition from any "evaluate bolus" mode back to the starting point of "Deliver basal rate closed-loop" mode)
The purpose of this transition from the "Evaluate correction bolus" mode is to stop learning the ISF and go back to learning the basal rates. The purpose of this transition from the "Evaluate prebolus" mode is to stop learning the C/I ratio and go back to learning the basal rates.
The logic of this transition is: IOB <5%
The process activation table[]
If you have come this far with a basic grasp of the states and transitions in our artificial pancreas algorithm, then you are going to love the process activation table, or "PAT." The PAT gives a list of the software capabilities that should be active depending on the present state or mode that our state machine is in at that time.
As we get further into the Hatley-Pirbhai Functional Decomposition, we will learn to think of software capabilities as a real-time processes and therefore will be documented in a series of process specifications or "p-specs". The PAT then tells us when to execute the p-specs to achieve the algorithm behavior that we want. The state machine and PAT work together to activate and deactivate the right software functions according to the natural observed states of the insulin pump and CGMS technologies.
Each software capability can be functionally described for now. The capabilities are:
"Deliver insulin" Capability[]
While the insulin pump will have the capability to deliver insulin almost all the time, we will deny it the ability to deliver insulin in the "Hypo Detected" state.
"Perform Diagnostics and Data Checking of CGMS Data" Capability[]
This is a diagnostic evaluation of the hardware and data from the CGMS. The CGMS data must be valid in order to control the insulin pump with it. When invalid, the pump will be operated open-loop. This capability is "on" all the time, so it is not really necessary to discuss it here in the context of the state machine.
"Deliver Basal Rate by Closed-loop CGMS" Capability[]
This capability will be used in several modes when we want to control the basal rate of the insulin pump in response to the blood glucose data from the CGMS. A baseline variable that is a function for time of day will be used as target value. CGMS feedback of the actual BG value will be used to generate an error signal. This will be filtered and adapted into a basal rate command to the insulin pump. The maximum and minimum basal rates will be bounded to avoid extremes.
"Learn the Basal Rate" Capability[]
When the loop is closed for controlling the basal rate, then the error signal generated from CGMS feedback will also be used to learn the desired basal rate for the recent time of day. If the error signal indicated that the basal rate was low, then the basal rate will be nudged in the direction that minimizes the error signal for future use. The basal rate closest to the present time of day will be incremented by a adaptation rate variable, and the other basal rates for the 1 hour time period before and after will be adjusted by 1/2 of the adaptation rate variable.
"Deliver Fixed Basal Rate" Capability[]
When closed-loop control of the basal rate is not possible, then the basal rate will continue to be delivered in the pump's usual predictive pattern but will not be adjusted according to the actual BG feedback.
"Learn the C/I Ratio" Capability[]
Following a manual bolus and waiting a period of time for the insulin and carbs to enter the bloodstream, it will be possible for the algorithm to evaluate the insulin balance in the person. If the calculated "insulin on board" is less than the insulin needed to bring the blood sugar level back to the target value, then the basal will be increased until the integral value of the basal increase is equal to the insulin shortage. During this event, it is also possible to evaluate the carb-to-insulin ratio that was used to deliver the pre-bolus. If the C/I ratio is found to be smaller than needed, then the C/I ratio for that meal or time-of-day will be nudged in the direction it needs to go for future use. Multiple C/I corrections will fix a bad ratio such that the pump algorithm will "learn" the actual C/I ratio and then track it as it changes. It is possible to determine the correct ratio and fix it immediately, but that would be undesirable: a carb measurement error is equally likely to cause a high blood sugar as is insufficient insulin calculated from a small C/I ratio. Carb measurement errors however will vary from overestimating and underestimating (the second being more likely). We will allow the rate to adapt in the negative direction to be greater than the rate it adapts in the positive direction. A slowly adapting variable for C/I will work much better in this situation.
"Learn the ISF" Capability[]
As in the "learning" of the other insulin pump control variables, the insulin sensitivity factor, or ISF, can be adapted when evaluating the response to a correction bolus. When the bolus is about half complete relative to the duration of action for that insulin, it is possible to evaluate the balance between the insulin-on-board and the blood sugar value, and close the loop with CGMS to control the basal rate. The error signal generated by the CGMS feedback can also be used to "nudge" the ISF that was used to predict the correction bolus so that the ISF is more accurate in the future.
"Deliver Manual Bolus" Capability[]
This is the open loop predictive capability to deliver a food bolus and then wait for the opportunity to close the loop again. The manual bolus is a prebolus started by entering grams of carbs about to be eaten and the present BG value, and then allowing the pump to calculate the insulin doses and deliver it to the pump controls.
"Deliver Correction Bolus" Capability[]
This is the open loop predictive capability to deliver a correction bolus and then wait for the opportunity later to close the loop again. The algorithm assumes we will want to do this every time the artificial pancreas system is started up (after a sensor change for example) where they may be some hyperglycemia to resolve.
"Deliver Adaptive Bolus" Capability[]
This is how we deliver the adaptive bolus: first deliver the "spike" proportionally to the peak of the glucose spike, hold steady the rate for the extended bolus according to the derivative of the BG data, and then control the basal rate per the integral function of the BG data until a balance between insulin-on-board and BG data is achieved. Someday, this might become an automatic bolus, but for now will be used only in the missed bolus mode. The user will be prompted when a missed bolus occurs, and then the adaptive bolus will be delivered only when acknowledged and when it's not bedtime or sleep time.
"Calculate IOB" Capability[]
This is the capability to determine how much insulin already infused into the patient is still active in the bloodstream and has yet to act on the blood sugar level. We can estimate it with the 7-4-1 rule: at one hour, past the bolus, there is 70% remaining; at 2 hours, 40%; and at 3 hours only 10%. This is for specific to Novolog, and is one of the variables that will be controllable from a CDE when setting up the pump for an individual patient. This capability is "on" all the time, so it is not really necessary to discuss it here in the context of the state machine.
"Display Basal Rate" Capability[]
The pump (or artificial pancreas) will the the ability to display information on a screen and for the user to select among the information they wish to see. However, when the user has not asked for anything in particular or has not touched the pump in a while, then the pump will display a default value. This default is chosen to help the user understand what mode the pump is presently in. Thus there is a map of what is displayed versus what is the mode. The basal rate will be the default display in a few modes. The rate will be displayed in units per hour with a resolution of 0.05 units.
"Display IOB" Capability[]
The pump will have the capability to display its calculation of insulin-on-board. In some software states, this will be the default display, although any data can be selected for display by the user.
"Display Closed Loop Heartbeat" Capability[]
This is just an idea, and many folks are bound to dislike it for medical or legal reasons. The idea is to show the user when closed loop control is being performed, since this will be confusing to the pump user. Consider a typical prebolus for example: the loop will be closed in the "control basal rate with CGMS" mode; then the loop is open when in the "deliver prebolus" mode; then closed again in the "monitor the prebolus" mode while it waits to go back to the first mode. The idea is simply to show the CGMS data right now, but to flash it on the screen instead of a solid or non-flashed display. The same data would be not flashed when it is not controlling the pump. The flashing technique will be to flash it with a 2/3 on, 1/3 off ratio with a very slow flash rate of perhaps 2 seconds on, 1 off, or a little slower. This will seem like a mantra, beating slowly and offering a very comfortable feeling to the user when the loop is closed. The reason we do this is to gain the extra diagnostic capability of having the user know when the loop is closed and can evaluate that previous events had occurred as expected.
"Display BG Trends" Capability[]
At times the default display will be the CGMS data and trends. This will include the usual things: most recent blood glucose value with a moving average, an icon to show an increasing trend or decreasing trend, and a status indicator (such as a fast flash rate to draw attention to out-of-range conditions), and predictions and warnings of pending hypoglycemia.
"Display Hypo Warning" Capability[]
When the pump is halted because a severe hypo is detected, then we want to show emergency information and emergency contact information that the user could input via their home computer. A beep or vibrate to alarm or draw attention is also desired. BG values may also be helpful on the display if they are available in this condition.