Developer blog

November 30, 2022

Properties (2)
The hidden state of hidden lines

Last time I introduced you to SFont, a simple yet useful representation of partially defined ("mixed") fonts:

struct SFont {
    std::optional<MyString> m_ostrName;
    std::optional<int> m_onSize;
    std::optional<bool> m_obBold;
};
Listing (1)

In our teminology SFont is called a "property", and its members m_ostrName, m_onSize and m_obBold are called "aspects" of the property. We discussed how SFont can be seen as representing a set of fonts, or a subspace in the space of all possible fonts, and leveraged those insights to define some useful methods and functions.

The whole point of that last blog article was to provide you with some concepts and notions that would help you design your own properties, not just fonts. In our software, we have a lot of properties that follow the same pattern: fonts, colors, fills, lines, bullets, markers (used in scatter charts and line charts to mark data points), and even precisions (formatting of decimal numbers). Today we will find out how line formats are different from fonts, and how we can amend our simple data structure to make it fit for line formats. Let's start with a straight-forward definition of SLineFormat, based on PowerPoint's LineFormat API:

enum MsoLineDashStyle {
    msoLineSolid,
    msoLineDash,
    ...
    // Refer to Microsoft Office VBA documentation for the full
    // definition of MsoLineDashStyle supported in PowerPoint.
}; 

enum MsoLineStyle {
    msoLineSingle,
    msoLineThickBetweenThin,
    ...
    // Refer to Microsoft Office VBA documentation for the full
    // definition of MsoLineStyle supported in PowerPoint.
};

struct SLineFormat {
    std::optional<bool> m_obVisible;
    std::optional<int> m_onWeight; // line thickness
    std::optional<MsoLineDashStyle> m_omsolinedashstyle;
    std::optional<MsoLineStyle> m_omsolinestyle;
    // add MsoLineCapStyle, MsoLineJoinStyle, ... and some
    // representation of fill or color as needed
};
Listing (2)

Seems simple enough, but what's the meaning of the dash style of an invisible line? How is a "visible" line with weight 0 (zero) different from an "invisible" line? Meet the concept of hidden state: We define hidden state as state (information) that is represented and maintained in a data structure, but is not discernible by the user. Everybody who designs data structures frequently meets hidden state, whether they like it or not. In many cases it is tempting to ignore hidden state in favor of some superficial simplicity and apparent symmetrical beauty, or simply because the developer failed to recognize what later turns out to be hidden state.

Hidden state is not necessarily a bad thing: Consider typing text into a PowerPoint textbox, in some deliberately chosen font. When for some reason you decide to re-write your text, a likely first step is to delete the existing text and return to an empty textbox. An empty textbox does not have any visual representation of font (leaving aside the size of the blinking cursor) and yet you expect that when you start typing, your text will be using the same font that was used before. That behavior is useful, and implementing it requires a deliberate usage of hidden state.

In most scenarios though, hidden state is more confusing than helpful. Typically, hidden state surfaces in the form of remnants of a data model's history. In our product domain, if we are not careful about hidden state, two charts may look identical "pixel-wise", but may exhibit very different behavior when the user, e.g., changes the underlying data. This is particularly irritating if the user received those charts from a third party and has no way of knowing how they were created.

Instances of "useless hidden state" are often referred to as "redundancy", and a general rule of thumb for the design of data structures is that redundancy is bad and should be avoided. Did we overlook any hidden state in SFont? No we didn't: All aspects of our definition of the SFont property are independent from each other. Each aspect is meaningful in its own right, regardless of the values of the other aspects. Borrowing a notion from vector spaces, we say that name, size and boldness of a font are orthogonal to each other.

Getting back to SLineFormat, we find that there are multiple representations of an invisible line. Given that all invisible lines look the same to the user, we can safely say that this is an instance of hidden state. At this point we need to decide if this particular hidden state is useful or harmful. I can immediately name some common use cases where an ambiguous representation of an invisible line is harmful:

  • When displaying a list of available line formats in the user interface, we want to show at most one instance of an invisible line format in the list. An invisible line format must compare "equal" with other invisible line formats for this purpose.
  • When displaying the current line format of a multi-selection of lines, none of which are visible, the user interface should reflect "invisible" and not "mixed", see method SFont::Union(SFont).
  • When determining whether the application of one line format on top of another line format would have any effect, we should not come to the conclusion that applying an invisible line format to another invisible line format results in a meaningful change, see method SFont::IsSupersetOf(SFont).

At the same time I find it hard to come up with any use case that would justify the maintenance of hidden state in an invisible line format. Thus, for the sake of this article, let's say that SLineFormat should not contain any hidden state.

How can we achieve that? For a start, there is a redundancy in members m_obVisible and m_onWeight. Either of them can indicate an invisible line format, so ideally only one of them should be needed. Obviously, m_onWeight is indispensible to represent a visible line format, but m_obVisible does not contribute any information that isn't also available from m_onWeight, thus m_obVisible should be dropped:

struct SLineFormat {
    std::optional<int> m_onWeight; // line thickness, 0 means invisible
    std::optional<MsoLineDashStyle> m_omsolinedashstyle;
    std::optional<MsoLineStyle> m_omsolinestyle;
}
Listing (3)

Microsoft's MsoLineDashStyle and MsoLineStyle enums don't have values for "invisible" or "no line", but there are still redundant representations of an invisible line. How is that? Well, if a line format has m_onWeight==0, i.e., it represents an invisible line, m_omsolinedashstyle and m_omsolinestyle may still have values. Thus there are a lot of different combinations of values in the members of SLineFormat, that can all represent "no line".

How can we get those meaningless combinations of values under control? There are at least two possible ways:

  1. We can ignore whatever values there are in m_omsolinedashstyle and m_omsolinestyle whenever we encounter m_onWeight==0 (see listing 4),
  2. or we can ensure that whenever m_onWeight==0, the other members always have one and the same unique value (see listing 5).
#define IS_EQUAL_ASPECT(member) ( \
    static_cast<bool>(lhs.member)==static_cast<bool>(rhs.member) \
    && (!lhs.member || *lhs.member==*rhs.member) \
) // same as for SFont

bool SLineFormat::IsInvisible() const& noexcept {
    // Note that !IsInvisible() does not necessarily mean "visible":
    // If !m_onWeight, then visibility is undefined/mixed.
    return m_onWeight && 0==*m_onWeight;
}

bool operator==(SLineFormat const& lhs, SLineFormat const& rhs) noexcept {
    return IS_EQUAL_ASPECT(m_onWeight)
        && (
            lhs.IsInvisible()
            || (
                IS_EQUAL_ASPECT(m_omsolinedashstyle)
                && IS_EQUAL_ASPECT(m_omsolinestyle)
            )
        );
}
Listing (4)
void SLineFormat::AssertInvariant() const& noexcept {
    if( m_onWeight ) {
        if( 0==*m_onWeight ) {
            std::assert( !m_omsolinedashstyle );
            std::assert( !m_omsolinestyle );
        } else {
            std::assert( 0 <= *m_onWeight );
        }
    }
    // For more details about how we deal with unexpected conditions in our
    // code, watch Arno's talk "A Practical Approach to Error Handling".
}

bool operator==(SLineFormat const& lhs, SLineFormat const& rhs) noexcept {
    AssertInvariant();
    return IS_EQUAL_ASPECT(m_onWeight)
        && IS_EQUAL_ASPECT(m_omsolinedashstyle)
        && IS_EQUAL_ASPECT(m_omsolinestyle);
}
Listing (5)

While ignoring the values of irrelevant aspects sounds simple and most canonical in theory, in practice it turns out that ensuring specific values for meaningless aspects allows for cleaner, simpler, more canonical code. Clean, simple, canonical code means fewer bugs to begin with, and better maintainability, therefore we mostly use the latter approach in our software. In particular, due to the way our properties are designed, each aspect conveniently has an "undefined" (std::nullopt) state, anyway. Using the "undefined" state for this purpose saves us from dealing with arbitrary magic values (which would also work in principle, of course).

With this basic understanding of dependent (non-orthogonal) property aspects, let's look at the implementations of some other key methods and functions for SLineFormat:

#define SET_ASPECT(member) if(rhs.member) member = rhs.member // same as for SFont

void SLineFormat::operator<<=(SLineFormat const& rhs) & noexcept {
    SET_ASPECT(m_onWeight);
    if( IsInvisible() ) {
        m_omsolinedashstyle = std::nullopt;
        m_omsolinestyle = std::nullopt;
    } else {
        SET_ASPECT(m_omsolinedashstyle);
        SET_ASPECT(m_omsolinestyle);
    }
    AssertInvariant();
}

SLineFormat operator<<(SLineFormat lhs, SLineFormat const& rhs) noexcept {
    lhs <<= rhs;
    return lhs;
} // analogous to SFont
Listing (6)

With (6) we can now use operator<<(...) to write chained expressions, just as with SFont, which are simple, elegant and, as we will see below, wrong (quote attributed to H. L. Mencken). For the sake of an example, let's look at our product domain again: We have a global default line format for visible lines, and we have a hierarchy of partially defined default line formats for specific purposes. How do we compose the default for the outline of a highlighted segment in a bar chart?

auto const lineHighlight = lineDefaultGlobal
    << lineDefaultSegment << lineDefaultHighlight; // WRONG!
Listing (7)

This expression worked great for SFont, what's wrong with using it for SLineFormat? Let's assume we start out with some sensible global default line format. Some modern styles for charts use colored areas without outlines, so lineDefaultSegment may be set to the unambiguous invisible outline. Highlighting a segment may justify a particularly thick outline (maybe even a red one, but we don't support color in our toy example), and it shall use whatever MsoLineDashStyle and MsoLineStyle is defined in the global default. Thus, our defaults may look like this:

SLineFormat const lineDefaultGlobal{
    /*m_onWeight*/ 6,
    /*m_omsolinedashstyle*/ msoLineSolid,
    /*m_omsolinestyle*/ msoLineSingle
};

SLineFormat const lineDefaultSegment{
    /*m_onWeight*/ 0,
    /*m_omsolinedashstyle*/ std::nullopt,
    /*m_omsolinestyle*/ std::nullopt
};

SLineFormat const lineDefaultHighlight{
    /*m_onWeight*/ 12,
    /*m_omsolinedashstyle*/ std::nullopt,
    /*m_omsolinestyle*/ std::nullopt
};
Listing (8)

If we feed the values from (8) into expression (7), the result is a partially defined line format: {/*m_onWeight*/ 12, /*m_omsolinedashstyle*/ std::nullopt, /*m_omsolinestyle*/ std::nullopt}. Obviously, this is not useful: In order to display a line around a highlighted segment, knowing the thickness is not enough. We need some well-defined MsoLineDashStyle and MsoLineStyle, too. Where exactly did msoLineSolid and msoLineSingle of lineDefaultGlobal get lost? The answer is: Associativity.

Operator precedence rules of C++ state that << is evaluated from left to right. If we hit one invisible line format in our chain of << operations, then according to (6) we clean out all hidden state. When we then hit another line format, that is partially defined visible, we cannot recover the missing aspects from the left-most operand. This can easily be fixed: We need to evaluate the chain from right to left:

auto const lineHighlight = lineDefaultGlobal
	<< (lineDefaultSegment << lineDefaultHighlight); // correct
Listing (9)

That's it for today! If there are only two things that you take away from this article, it should be that whenever you design a data structure, be aware of hidden state. And whenever you encounter hidden state, make a conscious decision how you want to deal with it, because that decision will affect the complexity of your code and the pitfalls you'll encounter when using your data structure. As always, if you have any feedback, don't hesitate to get in touch!

— by Volker Schöch

Do you have feedback? Send us a message at devblog@think-cell.com!


November 23, 2022

Properties (1)
Modeling the universe of fonts

Let's talk about a useful data structure for fonts. For the sake of this article, let's assume that a font consists of just a font face, a size and a flag for boldness. Turns out that even this simplified view of fonts is very useful in many use cases we encounter in our software:


struct MyString; // your favorite string representation

struct SFont {
    MyString m_strName;
    int m_nSize;
    bool m_bBold;
    // add italic, underline, strike-through, baseline offset,
    // spacing, allcaps, ... as needed
};
Listing (1)

In our PowerPoint add-in, we use a font representation like this for many different purposes, e.g.,

  • displaying the font settings of selected text,
  • applying font settings to selected text, in response to some user interaction,
  • applying default font settings to initialize some text that was inserted by our add-in.

This sounds easy enough, but there are some practically relevant limitations:

  • How can we represent a mixed font selection, e.g., where only some of the selected text is bold?
  • How can we represent a font for application when, e.g., the user wants to change only the size while leaving the boldness alone?

For display, we could use std::optional<SFont>, with std::nullopt representing a mixed font. This would work in principle, but even if only boldness is mixed in the selected text, and size and name are consistent, we could only show "Mixed font" without being able to display the size or the name. Not the greatest user interface on earth.

For application, we could use separate functions for the name, and the size, and the boldness setting. Given that in most practically relevant software, the user request for a font change would probably be processed by a hierarchy of nested function calls, the complexity of our SFont struct would proliferate throughout the entire call hierarchy. Want to add support for italic? Add a call tree of SetItalic(...) functions. Not the greatest software architecture on earth.

To solve both problems (and more, as we will see below), let's try to use std::optional inside of SFont:

struct SFont {
    std::optional<MyString> m_ostrName;
    std::optional<int> m_onSize;
    std::optional<bool> m_obBold;
};
Listing (2)

Let's call SFont a "property", and let's call its members m_ostrName, m_onSize and m_obBold "aspects" of the property. Aspects may be "mixed" or "undefined" (std::nullopt).

How does this solve our problem for display? When picking up the font aspects from text selection, we can fill SFont with well-defined information for aspects that are consistent throughout the selection. Aspects that have varying values within the selection can be represented by std::nullopt. When displaying the resulting font, we can display the values of consistent aspects, while for inconsistent aspects, we can show some representation of "mixed": "Arial ... pt bold", "... 12 pt" (implicitly non-bold) or "Arial 12 pt (bold)" are much more meaningful to a user than just "Mixed font".

How does this solve our problem with application? Whether we want to set size, boldness or name, or any combination thereof, we now pass an SFont object (or a reference thereof) through the entire call hierarchy. We call this a partially defined font. Functions that are just passing it on, do not have to care about which aspects of SFont actually carry values and which are std::nullopt.

With partially defined fonts, we can do some interesting things. For instance, in our software, we have hierarchical default settings: When determining the default font for a sum label in a stacked chart, we start with a fully defined global default font, which we infer from the PowerPoint Master Slide. We then apply, e.g., a general default font for chart labels, and finally a specific default font for sum labels. Except for the global default font, all fonts can be partially defined, so the chart default font may set the font size to 10 pt while leaving name and boldness alone, and the sum label default font may set the font to bold without affecting size and name. We write this neatly as:

auto const fontSum = fontDefaultGlobal << fontDefaultChart << fontDefaultSum;
Listing (3)

To facilitate this expression, we define the following operator overloads:

#define SET_ASPECT(member) if(rhs.member) member = rhs.member

void SFont::operator<<=(SFont const& rhs) & noexcept {
    SET_ASPECT(m_ostrName);
    SET_ASPECT(m_onSize);
    SET_ASPECT(m_obBold);
}

SFont operator<<(SFont lhs, SFont const& rhs) noexcept {
    lhs <<= rhs;
    return lhs;
}
Listing (4)

You may argue that you don't want to overload operator<< with functionality that is semantically unrelated to bit shifting. You are free to rename SFont::operator<<=(...) to, e.g., SFont::Set(...), but you are trading it for the conciseness of expression (3). Also, when using operator<< for this purpose, there is a potential issue with operator associativity, and we'll get to that in an upcoming blog post.

For now, while we are at it, let's look into some other useful methods for SFont. How can we pick up mixed font from selected text? If only we had a method to create something like a "union of fonts"...

#define UNION_ASPECT(member) \
    if(member && rhs.member && *member!=*rhs.member) member = std::nullopt

void SFont::Union(SFont const& rhs) & noexcept {
    UNION_ASPECT(m_ostrName);
    UNION_ASPECT(m_onSize);
    UNION_ASPECT(m_obBold);
}

void Union(std::optional<SFont>& ofont, SFont const& font) noexcept {
    if( ofont ) {
        ofont->Union(font);
    } else {
        ofont = font;
    }
}

template<typename RngFont>
std::optional<SFont> CollectFont(RngFont const& rngfont) noexcept {
    // see our public domain range library at https://github.com/think-cell/range
    return tc::accumulate(
        rngfont,
        std::optional<SFont>(),
        [&](auto& ofontAccu, auto const& font) noexcept { Union(ofontAccu, font); }
    );
}
Listing (5)

We are using std::optional<SFont> again, but this time it serves a different purpose. As the function name Union(...) suggests, we like to think of properties and aspects in terms of set theory. Our universe is the set of all possible fonts that could be represented by SFont. An SFont object represents a subset in this universe: If all members are well-defined, it represents a singleton. If no members are defined, the SFont object represents the universe. If a font has name "Arial" and size "12 pt" with boldness undefined, it represents the set with the two members "Arial 12 pt non-bold" and "Arial 12 pt bold".

Now that we are thinking in terms of set theory, we can say that by making some aspects undefined, method SFont::Union(...) enlarges the subset represented by *this. Specifically, it removes all aspects that *this and rhs do not agree upon. If there is no agreement at all, the resulting subset is the entire universe. But wait, if we want to iteratively calculate the union of n singleton sets, as is the case in our text selection example, how do we start? An accumulating iteration needs to start with an identity element of the respective operation. We conveniently use std::nullopt as an identity element for our Union(...) function. The identity element for union is the empty set and thus you can think of std::nullopt as the empty set in this example.

With std::nullopt serving as a convenient, generic identity element for any accumulating algorithms, and with the base type X for std::optional<X> being implicitly provided as the range value type, we can wrap this approach into a generic algorithm. We call it tc::accumulate_with_front(...), because the iteration starts with the first element of the range, rather than with an explicit start element. If the range is empty, std::nullopt is returned. Note that we do not need the helper function Union(std::optional<SFont>&, SFont) anymore, because it is implicit in the definition of tc::accumulate_with_front(...):

template<typename RngFont>
std::optional<SFont> CollectFont(RngFont const& rngfont) noexcept {
    // see our public domain range library at
    // https://github.com/think-cell/range
    return tc::accumulate_with_front(rngfont, TC_MEM_FN(.Union));
}
Listing (6)

The notion of SFont being a representation of a subset of fonts can also be useful when we want to know whether applying one font on top of another would make any difference: We can now phrase this question as an IsSupersetOf(...) predicate. Here are two implementations that are equivalent with regard to their results, although one uses more memory and more operations than the other:

#define IS_SUPERSET_OF_ASPECT(member) \
    (!member || rhs.member && *member==*rhs.member)

bool SFont::IsSupersetOf(SFont const& rhs) const& noexcept {
    return IS_SUPERSET_OF_ASPECT(m_ostrName)
        && IS_SUPERSET_OF_ASPECT(m_onSize)
        && IS_SUPERSET_OF_ASPECT(m_obBold);
}
Listing (7)
#define IS_EQUAL_ASPECT(member) ( \
    static_cast<bool>(lhs.member)==static_cast<bool>(rhs.member) \
    && (!lhs.member || *lhs.member==*rhs.member) \
)

bool operator==(SFont const& lhs, SFont const& rhs) noexcept {
    return IS_EQUAL_ASPECT(m_ostrName)
        && IS_EQUAL_ASPECT(m_onSize)
        && IS_EQUAL_ASPECT(m_obBold);
}

bool SFont::IsSupersetOf(SFont const& rhs) const& noexcept {
    return rhs==rhs << *this;
}
Listing (8)

While IsSupersetOf(...) in itself may already be useful in some situations, in our software we had one problem to solve that was closely related but a bit more tricky: We needed to extract the relevant information from one font relative to another, in order to store it, e.g., as a default or for quick access. A typical example would be a user applying an arbitrary partially defined font to some label that already has a font. We wanted to avoid storing any redundant, unnecessary information, because that would then inhibit our hierarchical font composition, see expression (3). Similarly, you may want to eliminate any unnecessary aspects from the font before application to (as it happens to be the case in our software) PowerPoint, because calls to PowerPoint can be expensive. Enter Minimize(...):

#define MINIMIZE_ASPECT(member) \
   if( IS_SUPERSET_OF_ASPECT(member) ) member = std::nullopt

void SFont::Minimize(SFont const& rhs) & noexcept {
    MINIMIZE_ASPECT(m_ostrName);
    MINIMIZE_ASPECT(m_onSize);
    MINIMIZE_ASPECT(m_obBold);
}
Listing (9)

Note: Minimize is not set difference!

You probably noticed that the result of the SFont::Union(...) method is not actually the set-union of the two subsets that were passed as parameters. Rather, it is the smallest set that contains both subsets and can be represented by our definition of SFont. That's the beauty of the data structure as presented in this acticle: It is very simple yet very useful. We achieve the simplicity by dropping the actual values of anything "mixed". For our practical purposes, that is a reasonable trade-off.

When trying to grasp this slightly peculiar behavior of SFont::Union(...), I find it helpful and illustrative to look at SFont in terms of an n-dimensional space (n==3 in our toy example), that is spanned by its aspects name, size and boldness. A fully defined font is equivalent to a point. If one aspect of the font is "mixed" or "undefined", the partially defined SFont object can be seen as a line in the space of all possible fonts. Remove another aspect, and you're left with a plane. When no aspects are left, the resulting SFont object represents the entire space. In this space, the operator<<(lhs, rhs) as introduced in listing (4) calculates the projection of lhs onto rhs.

If you have any feedback, I'd be glad to hear from you. Don't hesitate to let me know what you think about partially defined properties!

Next up: Applying the same ideas to the design of SLineFormat turns out to be less straight-forward than you may expect.

— by Volker Schöch

Do you have feedback? Send us a message at devblog@think-cell.com!


November 16, 2022

tc::change

Hello,

welcome to this blog of our development work at think-cell. It will be mainly about programming in general and more specifically about C++. Our platforms are mainly Windows and macOS, with a bit of web development sprinkled in. We will write about anything that comes to our mind, architectural decisions, little nuggets of wisdom we found or rants about bugs in 3rd party software that leave us frustrated.

At think-cell, we are writing mainly in C++, with some in-house Python scripts mixed in. We built our own in-house library which builds on top of Boost and the C++ Standard Library, and strives to follow the C++ Standard Library in conventions such as names, so that new users find it easy to get started. It is on GitHub and free to use under the Boost license.

I want to get started with a little utility that proved surprisingly useful for us and that has a bit more thinking behind it than is apparent at first sight.

We all know the concept of a dirty flag, which is set somewhere indicating some work to be done, and then queried and reset elsewhere where the work actually happens:

…
dirty=true;
…
if(dirty) {
    dirty=false;
    … do work …
}

Easy enough. There is a degree of freedom here though: do you reset the flag before or after the work is done? You may favor resetting it afterwards:

…
dirty=true;
…
if(dirty) {
    … do work …
    dirty=false;
}

This seems more expressive: you only say you are done with the work when you actually are. But is it practical? Clearly, if busy is not checked while doing work, it does not matter. What if it is checked? In particular, it could be that during the work, the part of the code doing work is being reentered. If you reset dirty early, the work will be skipped. If you reset it late, it won’t. What’s better?

…
dirty=true;
…
if(dirty) {
    … work part 1 …
    reenter_myself();
    … work part 2 …
    dirty=false;
}
At the time of reentrance, only part 1 of the work will be done, no matter how often you reenter the code doing work. Repeating part 1 on reentrance is likely at best not going to help, and at worst leads to an infinite recursion. To be correct, in any case, you must structure the work such that part 1 is sufficient for the code running inside the reentering code path.

If we accept that reasoning, resetting dirty early is always better. If part 1 is not sufficient, the code is incorrect anyway. If it is, we avoid redundant work and possibly an infinite recursion.

In our library available at GitHub, we packaged this insight into a little utility, tc::change, that is an assignment with the additional feature to return if something has changed:

…
dirty=true;
…
if(tc::change(dirty, false)) {
    … do work …
}

Besides for boolean flags, it also useful for other values where changing them entails some dependent work:

if(tc::change(size, screen_size())) {
    … consequences of screen size change …
}

We use tc::change sooo much. Try it!

— by Arno Schödl

Do you have feedback? Send us a message at devblog@think-cell.com!


Sign up for blog updates

Don't miss out on new posts! Sign up to receive a notification whenever we publish a new article.

Just submit your email address below. Be assured that we will not forward your email address to any third party.

Please refer to our privacy policy on how we protect your personal data.

Share