As long as there exists a bijection between whatever you choose as an index and the natural numbers starting from 0 it is fine. (I.e. the range of valid indices must be a countable set)
In your example that bijection could be:
3 -> 0
4 -> 1
5 -> 2
6 -> 3
This works for vectors as well, so why not have a range from (0,0) to (5,5) to index into an array arr?
You could write the function that does the mapping manually:
Just know that custom indexes / ranges are not automagically broken. Personally, I like how much easier it is to read the intent with custom indices.