I noticed there is also less tolerance in evaluating machine errors vs. human errors.
For example, there are thousands of car accidents a year which don’t make the news.
But if a single self driving car hits a traffic light, it makes the news.
Can you comment on this type of bias?
Someone invents and publishes a better ML technique, like attention or transformers. Next, a graduate student demonstrates using it to improve facial recognition by 5%. A small startup publishes an app that does better facial recognition. A government uses the app to study downtown walking patterns and endangered species, and after those successes, for court-ordered monitoring. A repressive government then takes that method to identify ethnicities. Repression, genocide result.
No one has made a huge ethical error at any incremental step, yet the result is horrific.
I have no doubt that Amazon will soon serve up a personally customized price for each item that maximizes their profits.
How can such “ethical creep” be addressed, where the effect is remote, from many small causes?
I’m as concerned about correctly functioning unbiased ML being used to serve harmful ends as with types of unintended bias.
Can team diversity mitigate some the biases? Are there any examples of this?
What approaches are there to break feedback loops where model outputs are used to re-train the model?
Specifically I’m thinking of payment fraud: if a model flags a transaction as fraudulent, that transaction will get flagged as fraudulent and blocked.On next training these fraudulent outputs will get used to train the next version… My question is what approaches exist to prevent the model from getting biased (aside from keeping thehuman in the loop)?
did stratification on utilization change the predictive factors for stroke?
How would you solve the stroke data bias problem? Is there any way to get rid of this bias? Especially with something like strokes that are pretty rare, how do you go through the process of fairly sampling who is actually having strokes?
Any extreme measures, like tracing every people during emergencies like COVID, should have an expiration date.
Agreed. I have seen some arguments being made that it is better to keep the sensitive features in the model so that you can measure and correct for the bias.
This video briefly talks about how to do this in case you are interested.
“The privileged are processed by people; the poor are processed by algortihms”
Wow. Looking forward to reading more from O’Neil!
But We have removed the human completely. There is no data that we are interested available here though…
This “huge ethical error” made the difference: A repressive government then takes that method to identify ethnicities.
“I stopped doing CV research because I saw the impact my work was having. I loved the work but the military applications and privacy concerns eventually became impossible to ignore.” - Creator of YOLO.
The task in this instance was predicting occupations (e.g. chef, gardener, athlete). Even though the flesh is mostly blurred out, the paper found that the occupations could still be predicted with minimal decrease in performance
It’s wild, this is the Twitter of the creator of the computer vision algorithm YOLO who decided not to contribute to the surveillance machine. https://twitter.com/pjreddie/status/1230524770350817280
Even more, can we expand to this question: How to make mutual feedback approaches for humans to reduce the bias of models, and models to reduce the bias of humans?
Should we have a Fast AI notebook which does solves some ML problem (basic one) in which we show how a person can systematically remove such biases? For example - How just removing the gender variable doesn’t remove the gender bias but multiple other things can be done to remove that bias.
