The Albanese government has laid out a list of consumer protection priorities for AI. (ABC News: Adam Kennedy)
Government agencies will have to follow a framework to ensure decision-making is "fair and accurate" when using AI.
The framework is among several priorities set out by the government in a list that includes a strengthening of privacy laws.
AI is shaping as a major focus at the Labor Party National Conference in Adelaide later this week.
Australia will tighten oversight of artificial intelligence in automated government decision-making and strengthen privacy safeguards as part of a broader push to allay concerns about the technology's risks to community safety.
A new list of AI consumer priorities published by the government on Sunday includes a promise to "better regulate" the use of AI in federal agencies like Centrelink and Services Australia, and revisit personal data protection laws.
Pressure has been building on the government to commit to human intervention in the administration of government schemes, particularly in social services, balanced with a desire to use emerging technology to boost productivity.
In May, Employment Minister Amanda Rishworth said humans would remain involved in the Jobseeker payment process as she announced an overhaul of the unemployment system.
Last year, the Commonwealth Ombudsman found almost a thousand jobseekers had their income support payments unlawfully terminated via automatic cancellations.
The government has also faced scrutiny over automated elements of the aged care home support program, as well as a planning tool in the National Disability Insurance Scheme that generates budgets for participants.
Earlier this year the ombudsman opened an investigation into the algorithm-based aged care assessment tool.
The government has also committed to reconsider protections for consumers and singled out retail surveillance pricing, the practice of farming data to set different prices for individual consumers based on behaviour, as a priority issue.