The numbers in your game are most likely relative. Their value only matters in relationship to each other. So having a card worth 2 victory points and a card worth 4 victory points could be functionally the same as having cards worth 4 and 8. Because of this you can have a lot of freedom in changing all of your numbers with minimal mechanical affect.
But what numbers should you use? Personally I like smaller numbers because the math is easier for people to deal with. So in the example above I would change it to 1 and 2 if possible.
But sometimes it’s not the percentage change between points, like doubling above, but rather the difference that matters. So 1 & 2, 2 & 4, and 4 & 8 are not the same if you care about the difference. In a system where the difference matters I would still lower the numbers by finding the lowest number, reducing it to 1, then subtracting the same from all other numbers.
Pay attention to the numbers you use, their relation to each other, and see if you can simplify them in anyway.