12345 is not secure: Proposing a PIN check feature for IRMA

The IRMA app can hold personal and sensitive information, such as a user’s name, citizen service number, and bank account number. The core functionality of IRMA is to disclose this information to requesting parties, e.g., to log in to a website. Naturally, it is crucial that only the legitimate user of an IRMA app can access the app’s contents and disclose these data. In order to guarantee this, the IRMA app uses a PIN that the user can set when opening IRMA for the first time. When users want to access IRMA in the future, they always have to re-enter their PIN. Likewise, disclosing information to other parties always requires users to enter their PIN. With this, the IRMA app follows examples that we know from, e.g., online banking apps and authentication apps (e.g., DigiD).

Thanks to the NLnet project IRMA made easy, the IRMA app received a security audit by the Radically Open Security (ROS) team in May 2020. One of their recommendations was to improve IRMA’s policy for PINs to make IRMA more secure. In this post, we pick up this recommendation, propose a new “PIN check” feature, and give insights into the reasoning behind the choices that led to this proposal.

The IRMA PIN

Since the big redesign of IRMA in 2020, the IRMA app is protected by a 5-digit PIN that users can choose freely. Users who want more security can select a longer PIN (up to 16 digits) via the app’s settings. The IRMA app thus allows for strong security. However, the fact that strong PINs can be set — unfortunately — does not mean that they will be used in practice. With no mechanisms in place to prevent this, some users likely opt for insecure PINs such as 00000, 11111, and 12345. The fact that users can choose such weak PINs was also noted in the ROS security audit. In its security considerations section, the report notes:

The app allows any PIN to be selected, e.g., 11111 or 00000, etc. Such weak PIN can easily be guessed/ enumerated.

Accordingly, in the recommendations section, the report states:

The policy for PIN selection should not allow such weak PIN selections. Repetition of numbers in the PIN could be disallowed.

We took this advice to heart. However, rather than simply disallowing repetitions, we dove a bit deeper into the subject matter and systematically explored how to handle PINs in IRMA.

What is a good PIN policy?

At first sight, the solution to the weak-PINs-problem might seem simple: As soon as a user chooses a PIN such as 00000, 11111, 22222 an error informs the user that the PIN is insecure and that they have to select a PIN without repetitions. However, on second thought, the issue is more complex, and many questions arise: How many repetitions are problematic? Is, e.g., 11112 acceptable? And if so, what about 11122? Are repetitions the only problem? What about PINs like 12345 or 54321? Should such sequences be prohibited as well? And are there other things — aside from repetitions and sequences — that we should look out for? Clearly, some research is needed to determine how to best help users choose safe PINs.

To get a better idea of how to improve IRMA’s PIN policy, we used two sources of inspiration: We consulted literature about PINs and looked at how Apple deals with weak PINs.

Characteristics of weak PINs

We found plenty of information about weak and secure PINs online. In particular, the paper This PIN can be easily guessed (Markert et al., 2020)¹ proved to be very useful. In addition, the thorough blog post entitled “PIN analysis” by Nick Berry² turned out to contain many valuable and surprising insights. Furthermore, the article 3 Simple Rules for a Good PIN on the trezor.io website helped us boil down the complex matter to simple rules that end users can easily understand and apply.

It became clear that ‘weak PINs’ come in many different forms. However, there are some properties that one can look out for to identify weak PINs. According to the consulted resources, the following PIN characteristics are very common and thus problematic:

• Ascending/descending sequences (e.g., 12345, 98765)
• Patterns, such as:
  • Repetitions (e.g., repetition of a single digit such as 11111³ as well as repetition of chunks, such as in the popular 4-digit PINs 1212, 1010 and the popular 9-digit sequence 420420420)
  • Number-pad related patterns (e.g., the four corners of the number pad or the middle row of the number pad for 4-digit PINs)
• Few unique digits⁴ (as e.g., in the popular 5-digit PIN 00700)
• Popular numbers (e.g., using 007 as known from James Bond, or 90210, which is the name of a popular TV series)

As the PIN Analysis blog post² points out, at least 4-digit PINs often also consist of:

• Memorable years (in particular, PINs using birth years starting with 19…)
• Dates (PINs using the MMDD format)

In addition, the consulted literature revealed many more interesting details about how people tend to choose PINs. The PIN Analysis blog post², for instance, mentions that people prefer even numbers over odd ones. It furthermore shows that people seem even less inventive with 5 digits than with 4 digits. Similarly, the paper by Markert et al.(2020)¹ finds that using 6-digit PINs instead of 4-digit PINs may decrease rather than increase security against a throttled attacker.

Naturally, these findings gave us much food for thought and prompted us to reconsider some choices made in IRMA. For instance, we wondered whether the default length of 5 digits is indeed ideal or whether it could be decreased to 4. However, as time and resources were limited, we decided to extend the current PIN implementation in IRMA with an additional PIN check feature rather than change the implementation altogether.

How Apple does it

To come up with a good PIN policy, we also looked at how others — and in particular Apple — deal with PINs. A user’s phone typically holds plenty of sensitive data. We assume that Apple — which positions itself as a company that cares about the privacy and security of its customers’ data — has put much thought into how to protect users’ devices and support them with choosing a secure PIN. To learn about Apple’s approach, we experimented with iOS ourselves and consulted the paper by P. Markert et al. (2020)¹, which also contains many details about Apple’s strategy.

When users first set up their new mobile Apple device, they are urged to choose a 6-digit PIN. The PIN that a user enters is then checked with a blocklist that contains PINs that — according to Apple — can be guessed easily (and thus should be avoided). If the user enters a PIN that is contained on the list, they receive a warning that the PIN can easily be guessed and get the option to either alter their PIN or proceed with this PIN anyway (just like Markert et al.¹, we will refer to this as ‘click-through’). A screenshot of this dialogue is presented below:

In addition to using a 6-digit PIN, iOS also provides additional options. Via the Passcode settings, users can change their passcode to a 4-digit PIN, opt for a custom numeric code or custom alphanumeric code (of self-chosen length) or turn off the passcode entirely. While all 4-digit and 6-digit PINs can be used with the above described ‘click-through’ option, users are prevented from proceeding with passcodes shorter than 4 digits/characters and forced to change their code. The screenshot below shows how Apple’s iOS reacts when users try to set such a short PIN. To distinguish such settings that force users to select a different PIN from click-through settings, we call this an enforcing setting.

The PINs blocked by Apple have been reconstructed by Markert et al. (2020)⁴, and the list of blocked PINs can be found on the project page that accompanies their paper. As Maximilian Golla’s presentation about Apple’s blocklist shows, the blocked PINs contain PINs with ascending/descending sequences, PINs with patterns (in the form aaaaaa, abcabc, abccba, ababab, ababa[0-9]) as well as other PINs that are considered common. (In fact, Maximilian Golla et al. were able to summarise the entire blocklist by combining a few simple rules with some common PINs.)

In addition to warning users about PINs on their list, Apple strictly limits PIN guessing-attempts of an attacker to 10, which also greatly affects security.

Considerations and choices

Based on the review of the above-mentioned literature and the exploration of Apple’s approach, we identified the following main considerations/choices:

• Whether to use (1) a blocklist with common PINs or (2) a rule-based approach that checks PINs for problematic characteristics or (3) a combination of rules and common PINs (like Apple does)

  • If a blocklist is used, what (and how many) PINs to include
  • If rules are used, what exact rules to include
  • If a combination is used, which combination

• What feedback to give to users when a weak PIN is detected

• Whether to enforce a PIN change when a weak PIN is detected or whether to allow users to click-through

As we want to ensure users do not get stuck when choosing a PIN, the answers to these questions depend on each other to some degree. If it is very difficult for users to pick a secure PIN because a huge blocklist is used, it might be good to allow users to click through. When using an enforcing setting, clear instructions should inform users on how to come up with a PIN that allows them to proceed. Hence, in an enforcing setting, simple rules that can be explained to users might be a good choice.

As mentioned, due to constraints in time and resources, we decided to extend the current IRMA implementation with an additional PIN check feature. This also meant that more fundamental considerations were declared out of scope for now. These include questions such as:

• What to use as the optimal default PIN length (and whether, for instance, to decrease the minimum PIN length to 4 digits)

• How to best throttle users, and whether, e.g., to limit attempts to 10?

• Whether to also support other factors (e.g., biometrics)

• Whether to add an additional factor (e.g., biometrics)

While we cannot revisit these considerations as part of this project, we propose to add the PIN check feature in a manner that allows the IRMA community to revisit the questions in the future and make changes iteratively.

Boundary conditions

Because we propose to extend the current IRMA implementation with an additional PIN check feature, we aim to find a solution that works well given the following existing context and constraints:

• IRMA requires a PIN. When a user forgets their PIN, they loose access to their IRMA app and have to start over with an empty IRMA app.
• Data in IRMA can be re-obtained. As of now, all data in IRMA can be re-obtained. This means that even when a PIN is forgotten, people can recover from this by starting over and re-loading all relevant data. While this takes effort, it is not as dramatic as loosing any data for good. However, this might change with future IRMA applications. (E.g., a voting card might only be obtainable once.)
• IRMA uses throttling. To be precise, users can enter an incorrect PIN three times in a row. After the third wrong attempt, a waiting time of 1 minute is enforced. With each subsequent wrong PIN entry, this time-penalty doubles. (This means that the first 10 wrong attempts can be made within a bit more than 2 hours, but for the initial 25 incorrect attempts, an attacker would already need approximately 8 years.) We can thus focus on guessing resistance against the first few guesses.
• IRMA does not allow users to recover forgotten PINs. As mentioned, when a PIN is forgotten, users have to start over with an empty app. It is not possible to reset or obtain the PIN via email or SMS (the organization behind IRMA does not know the PIN).
• IRMA uses 5-digits as the default PIN length but allows users to set a longer PIN via the settings menu. The chosen approach must work for PINs with a length ranging from 5-16 digits. When a PIN longer than 5 digits is chosen, an attacker does not know how long exactly the PIN is.
• IRMA is an app that explicitly focuses on privacy and security. This makes it reasonable to adopt a rather strict policy.
• IRMA can hold and disclose very sensitive information such as citizen service numbers. This also makes it reasonable to adopt a rather strict policy.
• IRMA aims to put the user in control of their data. One can argue that this entails allowing the user to make their own decisions about their data, even when these decisions are considered ‘bad’. If one takes ‘putting users in control’ very seriously, this could mean allowing users to choose weak PINs if users really want to do so.
• IRMA runs on the user’s phone. This means an attacker needs access to the user’s phone. It also means that users can protect IRMA further by setting a (different) secure PIN for their phone. However, we also have to keep in mind that when an attacker has access to a user’s IRMA app, they have the user’s phone and can likely find personal details, such as the user’s birthday, by searching through the phone. It thus is essential to consider preventing users from using meaningful personal dates as a PIN.

Criteria

We set out to find a PIN policy with a good balance between user experience on the one hand and security on the other hand. In particular, we aim to find a policy that scores well across the following criteria:

• Security. With respect to security, we focus on how well the policy protects users against throttled guessing attacks where the attacker knows the PIN policy. Ideally, the PINs flagged as insecure correspond to easy guessable PINs and the PINs that are not flagged are difficult to guess. While we focus on guessing attacks, we keep in mind that targeted attacks are also possible.
• User experience, and in particular:
  • Memorability of accepted PINs. We want to allow users to set PINs that they can remember.
  • Ease of coming up with an acceptable PIN. We want to minimize the attempts needed to select an acceptable PIN and the effort involved in this process.
  • Explainability of the policy. This relates to the previous item: If the PIN policy can be explained to users, it should be easier for users to come up with ‘a good PIN’.
• Feasibility. We aim to find a PIN policy that is feasible in terms of maintenance, implementation time and effort, and needed resources.
• Fit with IRMA. The proposed policy should fit the IRMA app and IRMA’s underlying principles and concepts (see boundary conditions and constraints above).

Recommendations

Constructing a good PIN policy is a challenge. As the paper by Markert et al. (2020)¹ shows, even Apple does not get it right. The following recommendations are presented to gain feedback from the IRMA community and as a possible first step towards improving IRMA’s PIN policy.

Lists versus rules

We have considered both blocklists or using a set of simple rules as well as combinations. As a first step forward, we propose to go with a rule-based approach. We suggest this for the following reasons:

• Feasibility: Constructing an effective list of ‘easy guessable’ PINs for codes of various lengths (ranging from 5-16) is a challenging and time-consuming undertaking. Constructing a set of PINs that furthermore adheres to the recommendations by Markert et al. (2020)¹ (and, e.g., would cover about 10% of the keyspace) would require too much additional work and is not possible within the constraints of this project.
• Explainability: assuming we use simple rules, it is easier to communicate to users what kind of PINs will be accepted. We can shift the focus from telling them that a certain PIN is bad to why it is bad and can give them concrete advice about how to do better. With this, we shift the focus from fighting the symptoms (weak PINs) to the underlying cause (the underlying PIN selection strategy). As a nice side-effect, using rules that can be easily communicated allows us to add an educative component and teach users about how to choose stronger PINs, thereby providing them with knowledge that can also be transferred to other applications.
• Ease of coming up with an acceptable PIN: because of the explainability of the policy, the ease of coming up with a PIN that is accepted is expected to be higher.

A disadvantage of a rule-based approach is that the resulting resistance against guessing attacks is likely lower than when using a well-crafted list that also includes common PINs that do not fit general rules. A rule-based approach (that, e.g., looks for sequences and repetitions) will not be able to spot popular PINs like “90210” (the name of a TV show). If IRMA uses simple rules, it will inevitably fail to warn users about some weak PINs. While this also holds for other approaches, these can be tailored much more granularly to match actual PIN selection behavior.

What rules to use

We have explored many different rules. We have started out with rules that took PIN length into account (at this point, we want to give a big thanks to Ronald for providing a great basis!). We then decided to simplify rules to gain explainability and make it easier to pick an acceptable PIN. For now, we recommend the following 2 rules:

A PIN must:

• Contain at least 3 different digits.
• Not contain ascending/descending sequences longer than 3 digits.

Note that the first rule also prevents PINs that solely repeat one single digit. (For 5-digit PINs, it means a number can not be repeated more than 3 times).

For now, we propose to use the same rules for 5-digit PINs and longer PINs.⁵

The suggestion to use these rules is mainly based on Explainability and Ease of coming up with an acceptable PIN.

What feedback to give

In line with the rules, we propose that IRMA urges users to use many different digits but avoid ascending and descending sequences.

Like Apple, we also use the PIN warning as an opportunity to explain why one would like to choose a secure PIN. However, unlike Apple, we give some concrete advice about how to select the PIN.

Concretely, we propose the following text for 5-digit PINs:

Insecure PIN

The IRMA app can contain sensitive personal information. Please use at least 3 different digits and avoid ascending or descending sequences in your PIN.

Hopefully, this feedback helps users who initially chose a weak PIN to adjust their PIN selection strategy and come up with a more secure PIN.

To force or not to force

We have considered whether it is better to force users to change their PIN or to only warn them and allow them to click through. For the initial implementation of the feature, we suggest a click-through setting. This is based mainly on the following considerations:

• Ease of coming up with an acceptable PIN: The fact that users can click-through ensures users can always come up with an acceptable PIN, as every PIN is accepted (if users choose to ignore the warning). It makes sure users cannot get stuck at this initial step in the onboarding.

• Security: The study by Markert et al.¹ did not find any evidence that enforcing a PIN change adds to security. As they point out, for an attacker, it is uncertain whether a user who got a PIN warning clicked through or not. (In contrast, in an enforcing setting, the attacker knows for certain that some PINs are not used.)

• Fit with IRMA - User control: One of IRMA’s goals is to put the user in control of their data. This means the designers must find a balance between protecting users on the one hand and giving them control on the other hand. By warning users but allowing them to make their own decision, we think IRMA can find this balance.

Does this policy work?

The big question now is whether these rules indeed work. Do they meet our criteria?

Let’s first look at security. A reason to believe that such simple rules can have a big impact on security is the fact that they would flag 15 out of 20 most used 5-digit passwords mentioned on the PIN analysis blog post² (see screenshot). Most importantly, the rule would flag the PINs 12345 and 11111. The PIN analysis blog found that ~22.8% (!!!) of people using an all-numeric 5-digit password selected 12345, and ~4.5% chose 11111. If the findings on this blog are representative of 5-digit PINs that users choose in IRMA, attackers are pretty likely to guess correctly when trying these common PINs. If the PIN warning gets users to avoid these common PIN choices — and choose uncommon PINs instead! — it already would be a big step forward. However, if the policy pushes users towards massively choosing the same PIN (e.g, 13579), the distribution of the most common PINs can get even more narrow than it is now, and the policy might actually backfire.

If we look at longer all-numeric passwords, this policy catches many but certainly not all popular choices.⁶ Interestingly, the effectiveness varies depending on the length of the PIN:

Even though the two simple rules catch many popular PINs, the proposed rules might not be strict enough yet. Our envisioned rules only flag 741 of the 2910 6-digit PINs flagged by iOS (Apple). This suggests that our rules miss quite some popular PINs. Furthermore, the envisioned rules only flag 1372 out of 100000 possible 5-digit PINs. In contrast, the paper by Markert et al.¹ suggests that blocklists (that can capture problematic PINs even more accurately than rules) should cover about 10% of the keyspace to be effective. Hence, it might be necessary to tighten the rules in the future.

Our design currently does not discourage the use of personal details such as birth dates. However, as the default PIN length is 5-digits, we believe preventing people from using memorable years and dates might be less of a priority.

The policy might worsen the user experience for those users who actually get to see the PIN warning. These users at least have to go through one additional step. We have chosen rules that are explainable and provide instructions about how to construct the PIN. This text is very short and concise. We thus expect it to be more difficult but still doable to come up with a PIN that adheres to the advice. How memorable these PINs will be is difficult to judge. Overall, we expect the user experience to be acceptable, also because users can always choose to click-through.

We believe the rules fit with IRMA’s underlying concept of protecting people’s privacy and giving people control over their data.

Implementing the policy is certainly feasible, and the basis for adding such a feature to IRMA has already been implemented in Dart by project member Ronald. This code is available to the IRMA-development team and only needs to be slightly adjusted to use the recommended rules. Before doing so, however, we recommend testing and possibly improve the policy.

Testing

As we have shown, getting the PIN policy right is challenging, especially because security and usability are so closely entangled. To ensure the new policy will be usable, we decided to create a mockup where the community can test the policy before it is introduced in IRMA. To this end, we have created the following website (see screenshot) where people can enter 5-digit PINs to see whether the PINs would be accepted and what kind of feedback the user would receive upon entering the PIN (currently, this feedback is presented in Dutch).

We recommend using the website to gather feedback from the IRMA community. The website can easily be adapted based on user feedback and thus also be used to arrive at a better PIN policy iteratively.

You can visit the website and try the rules yourself here: PIN check feature demo.

Just do it like Apple after all?

In our exploration of possible PIN policies, we had to make several tough decisions. Maybe surprisingly, we decided to deviate from Apple’s approach and develop our own suggestion (see above). In many ways, this feels risky. After all, Apple has likely invested many more resources to solve the problem than we will ever have. And even though Markert et al.¹ point out that Apple’s policy provides little or no benefit against throttled guessing attackers, their policy flags many problematic PINs that our simple rules miss. Thus, we also see possibilities for using Apple’s policy as a basis. In fact, we believe it would be interesting to compare our rules with Apple’s approach with comparative usability testing to see which policy users would prefer and how secure their resulting PINs would be.

In order to conduct such a comparative test, Apple’s rules would need to be obtained for 5-digit PINs. This could be done experimentally by checking which 5-digit PINs prompt a warning on the iOS PIN-change screen (similar to how Markert et al.¹ obtained the 6-digit PINs list). However, this will take quite some effort. Given that the list and heuristics used for 6-digit PINs have been made available by Markert et al., it might be an acceptable compromise to adapt the underlying heuristics for 5-digit PINs (have a look at their iOS_blacklist_heuristic_6digit.py file!).

If the IRMA team chooses to build on Apple, we suggest using PIN warnings with clear instructions about what to avoid. Based on the heuristics in Apple’s policy, we would recommend testing the following PIN warning whenever one of Apple’s general PIN rules is violated:

This PIN can be easily guessed

The IRMA app can contain sensitive personal information. Avoid repetitions, sequences and other patterns in your PIN.

For other PINs that are listed as problematic, custom messages could be designed. For instance, the following message could be tested for all common PINs that do not violate general rules:

This PIN can be easily guessed

The IRMA app can contain sensitive personal information. This is a popular PIN. Try to come up with a PIN that no one else can guess.

Conclusion and future work

The IRMA app currently accepts weak PINs such as 11111. The security audit of the IRMA app by ROS has pointed out this problem and has recommended changing IRMA’s PIN policy. With this blog post, we have reviewed existing knowledge about secure PINs and proposed a first step towards improving IRMA’s PIN policy. Concretely, we recommend a rule-based approach that requires at least 3 different digits to be used and checks for repetitions and ascending/descending sequences. We suggest not to enforce the policy and instead allow users to click-through.

Our proposed rules ensure users receive a warning when picking common PINs like 12345 and 11111. However, whether the policy helps depends on how people react to the warnings — on whether they change their PIN, and if so, what PINs they choose instead. Our main concern is that the feature might provide a false sense of security or, even worse: push users to an even more limited set of top choices that can easily be guessed.

We have shown that determining a good PIN policy is a challenging endeavour. We do not expect that this proposal is perfect. Likely, adjustments will have to be made based on user feedback. To this end, we have created a website where people can try the suggested PIN policy and the warning text we recommend. The website can be used to test and improve the proposed policy and text before implementing it. Furthermore, we suggest comparing our solution to Apple’s approach to determine the better choice. In general, we believe that PIN warnings should focus on helping users adjust their PIN selection strategy rather than on changing their specific PIN. This means the warnings need to give users information about a good strategy.

This blog post has focused on what can be achieved in the short term. In the long term, it would be great to conduct user research and construct a list of common PINs to more accurately warn users when selecting such PINs. It would also be interesting to test different forms of warnings and instructions to see how users best can be supported with choosing a secure PIN. In this regard, it makes sense to keep an eye on research literature. With respect to IRMA, it would be worthwhile to revisit more fundamental questions, such as whether to allow for biometric authentication. While this falls out of the scope of this project, we hope this will happen in the future.

P.S. Please do not use any of the PINs featured in this post.

---

1. Markert, P., Bailey, D. V., Golla, M., Dürmuth, M., & AviG, A. J. (2020, May). This PIN Can Be Easily Guessed: Analyzing the Security of Smartphone Unlock PINs. In 2020 IEEE Symposium on Security and Privacy (SP) (pp. 286-303). IEEE. ↩︎

2. "PIN analysis", blog post on the “DataGenetics” blog by Nick Berry September 2012 ↩︎

3. This form of repetition was mentioned in the ROS report. ↩︎

4. This issue goes hand in hand with repetitions. When, e.g., repeating the digit 1 for 5 times, on naturally only uses one unique digit. ↩︎

5. This policy can result in longer PINs being flagged even though they contain an ‘acceptable 5-digit sequence’. From a mathematical standpoint, this can be counterintuitive (the PIN comprises a sequence that was considered ‘good enough’ earlier and only has gotten longer, so how can it be flagged now?). It can also be weird for end users if they choose to make their current PIN longer. (Changing the PIN, e.g., from 11123 to 111234 will result in a warning.) However, we expect very few users to encounter this and believe sticking to the same rules for all PINs is the easiest for users when they need to come up with a PIN. We thus see this issue primarily as a theoretical problem and do not expect it to pose a huge issue for users in practice. ↩︎

6. We had initially considered including a rule that flags repetition of the same digit (e.g., PINs that use the same number more than 3 times in a row). However, based on the table with common all-numeric passwords above, we believe such a rule would add relatively little extra benefit. ↩︎