From algorithms that inform your music streaming platform and website chatbots to self-driving cars and predictive analytics, automated decision-making (ADM) is playing an increasingly integral role in online and other digital activities. But how is the data that inform the many applications for ADM being handled?
As part of Québec’s Bill 64 suite of privacy and data governance obligations (read our previous analysis on Bill 64 here), one set of obligations applies to ADM that is based on personal information. While Bill 64’s ADM requirements only come into effect in September 2023, businesses subject to Québec law should start acting now to be compliant with these ADM transparency and accountability rules.
Following the lead of regulators in Europe, South America, and elsewhere, Bill 64 provides for consumer rights to information and objection when their personal information is used to make decisions about them without independent human judgment. ADM processes can vary vastly in their complexity; from a simple binary choice based on whether a certain piece of information exists or not, to AI and the intricate algorithms that recommend certain content or search results.
When it comes into effect, Bill 64’s ADM provision—which will be section 12.1 of the Act respecting the protection of personal information in the private sector (Private Sector Act)—will establish transparency and accountability requirements for applicable ADM processes.
Notably, the provision only applies to ADM decisions that use personal information and are exclusively automated. Organizations must:
These transparency rights apply to individuals both within an organization (including employees) and outside it (such as customers).
While Bill 64’s ADM requirements are unlikely to stifle innovation or process efficiency in the long term, some significant up-front work will be required to inventory an organization’s ADM processes and review consents and privacy policies to ensure compliance.
Businesses should start by identifying the ADM processes that a) are exclusively automated with no human input on the outcome of the decision, b) use personal information as part of that decision, and c) are within the jurisdiction of the Private Sector Act (typically by virtue of a company’s operations in Québec or the ADM process involving personal information of Québec residents).
The inventory exercise should also consider the relative risk of each ADM process, as this will help triage the compliance workflow. The analysis should weigh sensitivity and volume of personal information involved, impact of the decision on the individual, existing documentation explaining how the ADM works, the extent of changes required to comply, and the ease of making those changes.
For each ADM process subject to Bill 64, notice of the automated decision must be provided to the individual. Notice can be given at the outset of an application process that involves ADM (for example when seeking consent to use personal information for the application) or when the decision is provided to the individual (for example when advising customers that they are eligible for a new product). It is not yet clear whether general terms and conditions can include the required disclosure or whether the ADM notice will need to be presented separately, but at a minimum the purpose of each ADM will need to be described. Organizations should not assume they can provide a general notice of automated decision making for various, unspecified purposes.
Companies that use ADM processes subject to Bill 64’s requirements will need to provide contact information that allows an individual to submit questions or complaints about the decision, how it was made, or the personal information used in the ADM. Individuals should also be able to submit requests to correct their personal information used to make the decision.
However, a business may also choose to build in automated processes to correct personal information used for the decision and/or to explain the reasons that led to a decision. Take the example of an ADM process that provides a free quote online. This tool can be built to allow an individual to alter (and thereby correct) the inputs to the quote system. In addition, it may be possible to design the results page to automatically explain the factors supporting the quote.
In order to meet these transparency obligations, companies will need to document the parameters used to make automated decisions. For more advanced ADM processes based on algorithms, this requires records of how the algorithm works, including the possible inputs and how they are weighted in the analysis. Bill 64 also requires that the information used to make a decision (automated or not) about a person is kept for at least one year following the decision. Businesses should therefore ensure that data retention mechanisms are in place for ADM processes.
Bias in ADM also remains a focus of regulators (particularly in the context of AI and machine learning). Some automated decision-making could be prohibited by provincial or federal employment, human rights or privacy legislation. To prevent bias and related complaints, businesses should document the design, controls, and testing in place to ensure results are accurate, fair, and appropriate, and the personal information used is not overly invasive.
While Bill 64’s ADM requirements only come into effect in September 2023, businesses subject to them should start acting now to inventory their ADM processes, update consumer disclosure and ensure appropriate record-keeping will be in place. Organizations should also build Québec privacy law compliance into their processes for implementing new ADM systems, such as by updating risk assessment and consent templates.
To discuss these issues, please contact the author(s).
This publication is a general discussion of certain legal and related developments and should not be relied upon as legal advice. If you require legal advice, we would be pleased to discuss the issues in this publication with you, in the context of your particular circumstances.
For permission to republish this or any other publication, contact Janelle Weed.
© 2023 by Torys LLP.
All rights reserved.