Runtime Change Propagation Control
The patented technique of Runtime Change Propagation Control (RCPC) provides the ability to stop the propagation of insignificant changes through the flow.
Motivation
A dependency management system based exclusively on timestamps, such as
make, assumes that any change to the timestamp of a file is a
change that needs to be propagated, even if the information contained in the file
has not changed significantly or has not changed at all, as happens when you use the
program touch on a file.
Figure 1.
In software development the comments in the source code do not affect the behavior of the executable code. If we have a consistent (completed) build process and we change a comment in a source file, we do not really need to recompile the code or to repeat the testing. However, if we are using a dependency system that is based exclusively on timestamps, the build will be considered inconsistent, with out-of-date and untested executables, until both compilation and testing are run again. This approach wastes compute resources when source code changes are only in the comments.
This example demonstrates that a dependency graph does not prevent unneccessary work from being performed. However, with the addition of run time control of change propagation, the extra work can be avoided.
Runtime Control of Change Propagation
A better system than depending on timestamps is to introduce a step that analyzes the change and tells the build system if the change should be blocked in the dependency graph. When change propagation is blocked, the system can suppress running the dependent steps.
The model of FlowTracer's Runtime Change Propagation Control is that an executable program used in the job sequence is assigned the responsibility to decide the change propagation case for a given file and to notify FlowTracer about it during the run. This program is customized to be the single agent that has that responsibility because it has specific knowledge about the meaning of the data.
Notification of FlowTracer can be done directly by including calls in the checking program to the appropriate procedures in the VIL library. Another way is to use a shell script and command line calls to helper programs that notify FlowTracer about the change propagation status.
Figure 2.
Imagine this program is called cleverstrip. It can reduce the new
version of the source file to a minimized form which is stripped of all the
comments, and can compare it to the minimized form of the previous version to see if
changes were only to comments. If the minimized versions are the same, then the
change was only to comments and change does not need to propagate. If the minimized
versions differ, then the change was to more than comments and the change should
propagate.
This approach is to create a derived file from the file that is to be analyzed. In the diagram of this situation, the source file to consider is file A. The derived source file is A'.
The analysis of the file creates a decision point in the dependency graph which is a
called a "barrier". This part of the graph is the dark segment shown at the
tip of the output arrow of cleverstrip. It marks that output file
as a "barrier file". The checking program is responsible for notifying
FlowTracer on the state of that "barrier file". Is it
open, allowing the change to propagate, or is it closed to stop the change from
propagating?
The original diagram changes with the introduction of this decision point. The logical steps change from the simple two step segment of "Source File" to "Compilation" into a four step segment of "Source File" to "cleverstrip" to "Source File Prime" to "Compilation".
If cleverstrip is fast and efficient relative to the compilation
and testing steps, computation time is saved while still maintaining integrity in
the compiled and tested release products.
Introducing runtime change propagation logic into a job sequence is a useful way to save compute resources. Using a derived file as the "barrier file" is a straight-forward way to implement the logic. Another way is to have the basic file become the barrier file. This is done by changing the job that creates that basic file to contain the runtime change propagation control logic.