Yes, organic material rotting in your water can definitely impact pH. I'd skim it out as best you can... just add it to the list when you clean/change out reservoir. Some sort of agitation is occuring one way or another..
Poorly buffered hydro fertilizer will fluctuate more wildly, too. Using RO water can exacerbate it... less solutes, more easily swayed... but again, properly buffered products shoud mitigate that.