This is something that has always been on my mind. When I was maybe in first grade, we had a big tank with a lot of goldfish. I wasn't really in charge of their care, but I fed them.
There was one, the biggest, that I named Alice. S/he (because I didn't know, I just assumed they were all female because I was a girl) and a few others would let me pet them when they came up to eat. She'd even let me pick her up out of the water and hold her for a few seconds. I thought this meant she liked me and trusted me, but I dunno.
At some point, some unknown disease killed them all. I always wondered if it because I'd picked her up and passed my human germs on to her. Is it bad to try and play with a fish like that? I recently saw this video gif of someone with their Red Devil, and it reminded me of what I did with Alice: http://giant.gfycat.com/GoodPleasedItaliangreyhound.gif
The fish looks like it's having an okay time, but is this dangerous for them in any way, shape, or form?