Hard legal stance: If you take something from somewhere else and create something new based around that thing, then you have created a "derivative work" and you might be in violation of copyright because of that. Creating derivative works is a right reserved to the original copyright holder.

Realistic stance: Technical people like ourselves like hard and fast rules. This is valid, this other thing is invalid. There are lines drawn in the sand, and you are one side or the other. Every question has an answer.

Sadly, this makes us terrible at law. Why? Because the law is not black and white.

Trying to define any set of hard and fast rules when it comes to legal matters is doomed to ultimate failure, because what is "legal" is really a matter of opinion. And it's the judge's opinion that matters. And you don't know who that judge is in advance. And he may not be fully versed in technical matters. And he might not even believe you when it comes to those technical matters, because somebody else is on the other side of the fence arguing against you, perhaps using what you think are false statements.

So there is no hard and fast rule for what is copyright infringement or not, and there never will be. All anybody can do is to give you guidelines. All anybody can do is to tell you what their personal opinion is. That opinion may be backed up by precedents, or it may not, but in the end it's the judge's opinion that actually matters, if it ever comes to that. And judges can decide any way they like.

So... is taking a small piece of something and building around it copyright infringement? Technically, yes, if you don't have permission. But, fair use might come into play, giving you an out. Or nobody might ever notice. Or nobody might ever actually care if the work is different enough. There's no way to predict the future, and there's no way to really predict the outcome of a trial on the topic. Safest bet is to make it yourself with your own images and code, or to use freely licensed images and code. And if you want to "steal" a large piece of work to mess around with, get permission first.