Checklists for Authors

A. Checklist for Anonymising Submissions

• Remove author and institution information from the cover page as well as from the acknowledgments section
• Clear meta-data in a word processor or PDF viewer/editor
• Replace institution information in the body of the text (including any approvals from IRBs or equivalent ethics boards/committees) with generic identifiers (e.g., “this research was approved by [anonymized] Institutional Review Board”; “we obtained approval from our university ethics committee”).
• Use third person for citations to own work
• Remove marks for institutional affiliation from images and supplementary materials (as much as possible).

B. Checklist for best practices on how to write a paper and best practices on responsible Machine Learning research

• Read the ACII Submission Guidelines.
• Make sure the paper’s contributions are clearly stated in the abstract and introduction.
• Make sure your claims in the paper match the theoretical and/or experimental findings.
• If you include theoretical results, state the full set of assumptions and the complete proofs of the theoretical results. If the complete proofs of the theoretical results are too long to be included in the main paper, they must be included in the supplementary material.
• If you run Machine Learning experiments:
  • Ensure you include and share all the necessary information to reproduce your results (e.g., hyperparameters, training splits). We encourage you to release your code and trained models if you are allowed to do so.
  • Make sure you provide evidence of the stability of your results. For example, run the experiments multiple times with different random seeds if you use random initialisation and provide error bars.
  • We encourage you to share the amount of computation and the type of computational resources used.
• If you run statistical inference:
  • Explicitly state which statistical test(s) you ran (e.g., Welch’s t-test, Mann-Whitney U-test, OLS regression, Pearson correlation), and ideally include your analytical code in the supplemental materials
  • Report all your p-values, including non-significant ones, and report them exactly unless they are smaller than .001 (e.g., p=.459, p=.003, p<.001)
  • Report effect sizes and confidence intervals (e.g., the mean of Group 1 was 0.4 SDs higher than that of Group 2, d=0.40, 95% CI: [0.30, 0.50])
  • Avoid causal language unless testing for causality (e.g., talk about X explaining or being associated with Y, rather than X causing/leading to Y)
  • Briefly discuss the plausibility of the assumptions of your model(s)
• If you use existing assets (e.g., code, data, models), properly cite the original source.
• If your work contributes new assets (e.g., code, data, models) and you can not release them, explain why you are not releasing the asset.

C. Ethical Impact Statement Checklist

• Please read through the Ethical Impact Statement Guidelines and this document on how to write an Ethical Impact Statement for a lengthier discussion of the details of how to write an Ethical Impact Statement.
• Fill out the Ethical Impact Checklist (this checklist) and note any items that do not apply or that you would like to elevate for discussion.
• If you conducted research with human subjects:
  • Include in your Methods section a short, clear summary of what instructions were given to the participant that may have influenced your findings. (Note that full study instructions may be too long to include; it is fine to omit lengthy instructions that are not likely to have influenced the study findings)
  • State if the study was IRB-approved, IRB-exempt, or not given to any IRB. Give the IRB approval number. In the case of “no IRB”, say why not
  • Describe how informed consent and/or assent were obtained from human subjects or explain why these were not applicable. (These should go in the Methods section of your paper.)
  • Explain how human subjects were compensated for their participation and how that compensation was determined. (These should go in the Methods section of your paper.)
• Discuss any potential negative impact of your work and strategies for mitigating these risks:
  • Can the research be used to deceive people? What steps could be taken to mitigate this?
  • Does or could the research contain bias against certain groups of people that could result in discrimination? Will it exacerbate already-existing biases (e.g., will it perpetuate gender or racial bias?)?
  • Can the research or technology described be used in applications that limit human rights or impact people’s livelihoods? For example, surveillance or limiting access to jobs, schools, etc.
• Discuss the limits of generalizability of your work.
  • For example, point out strong assumptions and discuss how robust your results are to violations of these assumptions.
  • Discuss the scope of your claims. For example, maybe you used a small dataset with poor demographic diversity. If that’s the case, you might want to discuss how the limited diversity of the dataset you used affects the scalability of your approach to larger and more diverse datasets.
  • Discuss the factors that can influence the performance of your approach. For example, an English affective speech-to-text system might not work properly for non-native English speakers.
  • How sensitive is the research to contextual factors? If the work is “generic”, e.g., a generic facial expression classifier, what are the considerations about generalizing to particular contexts?