Hon. Laurie A. Cumbo
New York City Council Majority Leader
250 Broadway, Suite 1833
New York, NY 10007
Re: Intro 1894-2020 - The Need For A Moratorium
Dear Council Member Cumbo:
We, the undersigned labor and civil society organizations, urge you to amend Intro 1894-2020 to more effectively address automated employment decision tools’ (“AEDTs”) impact on workers, particularly BIPOC New Yorkers. In lieu of the current bias audit requirement, we ask you to support a complete moratorium on AEDTs. AEDTs impact New Yorkers’ job opportunities, wages, work schedules, promotions, and termination. 1 These systems are too dangerous to be left unregulated, but Intro 1894 currently falls short of an effective remedy.
As written, Intro 1894 provides Silicon Valley a rubber stamp for AEDTs. While we need to regulate these tools, we must not rush a weak bill through. Our concerns, which were first raised to your office months ago, are outlined in detail below.
First, the bill’s definition of AEDTs omits many popular employment technologies. The definition must be strengthened to apply to any systems that are used to make employment decisions or to assist human decision-makers in the making of employment decisions, including applicant tracking systems, digital versions of psychological and personality assessments, among other automated procedures.
Second, and most critically, the bill’s bias audit requirements are vague and unenforceable. There is no consensus on what an effective bias audit would look like and no bias audit would mitigate all harms. Even so, the bill fails to define what a compliant bias audit must entail. Bias audits should at least include both vendors and employers and cannot apply only to the sale of the technology. In order for any bias audit to exhibit real-world impact, it must use data from employers which would be used by the tool, and a pre-sale audit requirement would not include testing with that data. Furthermore, limiting the audit requirement to pre-sale analysis not only leaves out oversight of tools already in use, but also oversight of tools used by companies like Amazon who develop their own in-house systems. The bill also does not specify that the audit be conducted by a third party. We cannot allow the companies selling these tools to make their own determinations of bias.
Third, the bill allows sale and use of biased AEDTs. For those companies that fail to comply with Intro 1894’s minimal bias audit requirements, the low penalties will simply become the cost of doing business for AEDT developers. Furthermore, Intro 1894’s reliance on agency enforcement and lack of a private right of action leaves even that minimal deterrence in question.¹ Workers should have the opportunity to seek redress for AEDT discrimination, as well as the attorneys’ fees to make such litigation possible.
Intro 1894 will require AEDT sellers to provide a veneer of due diligence, but nothing more. Moreover, it allows vendors to continue selling AEDTs that they know are biased.
We reiterate our opposition to Intro 1894 and ask that the Council amend the legislation to provide a categorical moratorium on AEDTs until a viable regulatory structure can be developed. New Yorkers deserve better regulation of the technologies that control their livelihoods.
Sincerely,
Adhikaar
AI Now Institute
Beta NYC
Center on Race, Inequality, and the Law at New York University School of Law
Electronic Privacy Information Center (EPIC)
Ethics in Technology
Fight for the Future
Freedom to Thrive
Make the Road New York
Mothers Against Wrongful Convictions
National Employment Law Project
S.T.O.P. – Surveillance Technology Oversight Project United for Respect
Emlyn Bottomley, Data & Algorithms in the Workplace: An Overview of Current Public Policy Strategies, U.C. Berkeley Labor Center (Nov. 2020), available at https://laborcenter.berkeley.edu/wpcontent/uploads/2020/12/Working-Paper-Data-and-Algorithms-in-the-Workplace-An-Overview-ofCurrent-Public-Policy-Strategies.pdf.