That is as logical an argument as saying that the universe is determined by physical laws. "Physical laws" are a description of patterns in what we have observed. When predictions based on those observed patterns get borne out consistently, the pattern gets called a"law". It's still a description of observed behavior, not the determining factor in that behavior, any more than a great betting system is a way of controlling how horses run. (This may be a terrible surprise for some folks.)
And just as many gambling systems are based on observing the patterns in random chance events, physics itself can be described the same way. A big enough mass of chaos begins forming patterns -- especially if observed by a human mind, which dearly loves to see patterns in things.