Reliance on any tool, regardless how simple or complex, carries the potential to lead to error - but 'doing it in our heads' carries risk as well. I'm not talking about a mistake in a calculation mind you - it's when people repeatedly go down the wrong path because there is a mental block or a blind spot. I see this a lot, and it's not a matter of intelligence.
The issues being discussed here are directly related to programming and that is a niche application of this technology. I suspect for most uses AI will be a great aid, both as a time-saver and in terms of needing to know less (but enough to apply the tool).
I speak out of experience with medical technology where there have been a great many efforts to incorporate AI into diagnostic tools for example. It very quickly becomes obvious that this can never be a fully automated process (or at least not in the near- or medium-term) because while the tools look nice and superficially appear to work, any level of scrutiny quickly reveals these are not perfect and the key is to recognise issues and know how to manipulate the tool to provide the correct answer.
For example I couldn't in a thousand years manually determine the motion of every voxel of a 3-dimensional cardiac ultrasound acquired at a rate of 80 Hz (i.e. 80 three-dimensional acquisitions per second), and if you wrote an app to track these, assuming you could, it would be so slow as to not be clinically relevant, but AI based technology does this for me in seconds. I simply determine the accuracy and intervene to improve the 3-dimensional deformation analysis, and the whole thing takes between 30 and 60 seconds (for comparison, a 'dumb' app for this about 20 years ago would take between 1 and 2 hours for a 20 Hz data set).
Specifically in the area of programming - a lot of programming is formulaic and for many algorithms there is a 'best way' to do things. Learning them all (ie. googling them
) is what we do now, but if the IDE had an AI tool for generating algorithms I for one would not be averse to this. Of course I'd review it and test it, but it would still save me time.
Does this make me lazy and reliant on the technology? Probably... but I'm not sure that's a horrible thing...