I already do this. I'm one of those people who like to use as much of my patterned paper as I possible can. I tend to choose my scraps by either the colour method where I pull out a main piece of paper(usually one with a multicoloured pattern) and then gather co-ordinating pieces to go with it. I then set these aside in a page protector ready to add a photo at some point.
My other method is to choose papers to match a particular picture I intend to use. Here I look for papers with colours already present in the picture I am using, paying particular attention to the mood I wish to create within the layout. For example these pictures from Pinterest show how you can pull out colours from a picture to best effect.
|These would create a layout of tranquility and calm.|
|This picture says happy and bright so the colours pulled from it reflect that positive feeling.|
|There is no denying this colour palette says cold and remote.|