Yes, cleaning copper out certainly matters. In precision rifles, I clean thoroughly, including copper, but seldom. I monitor and enter how many rounds of which type of ammo were fired in that rifle's log book. When a rifle is thoroughly cleaned, that too gets entered into the log book. I have best results (accuracy) when I foul the rifles with around 10-20 rounds before I really begin to compete, or test loads for accuracy. Usually, in my precision rifles, accuracy begins to fall off around 200 rounds or so. I'll first try a light cleaning with only Hoppe's, and shoot. If accuracy returns, I continue, if not, I will do a more thorough cleaning.
With rifles like AR15 that are used for practical rifle matches, or 3 gun shooting, I do not consider them to be precision rifles, so they are cleaned less often. I clean those rifles to get out carbon, unburned powder, re-lubricate, and inspect them. Copper is only cleaned out of the bore if I notice a significant decrease in accuracy, or at around the 400-700 round count. Less than 400-700 rounds and the rifles are cleaned with something like Hoppe's, and not cleaned to remove copper (something like Sweet's 7.62).
Whenever I clean to remove copper, I am very meticulous about removing any of the copper solvent, and insuring that none of the solvent is left anywhere in the rifle. That kind of requires the I re-clean with Hoppe's to remove the Sweet's, then re-lubricate the rifle, and insure that the bore and chamber are free of all solvents and oil. The final touch is running patches with rubbing alcohol through the bore and chamber to insure that there is no solvent or oil of any kind left in the chamber and bore, then using dry patches to insure all the alcohol is removed. Final process is to re-lubricate any parts requiring lubrication. Bores are not lubricated unless they are going into long term storage, in which case I put a little tag onto the trigger guard noting when the bore was oiled.
Having oil in the bore raises pressures, hence reducing accuracy as well as safety. Having oil in the chamber increases bolt thrust as the cartridge cases cannot grip the chamber walls adequately.
I realize this sounds like lots of work, but it really does help keep my rifle's working well, and makes it less likely that I will blow my rifle and/or my self up.