Seems nice, but beyond 6 points the polygons are hard to distinguish easily. Certainly not something you can quickly glance at in a rush. I'd say sticking to 5 maximum would be best.
I don't think empty for 0 necessarily makes the best sense. You wouldn't easily be able to see where the gaps are at a distance.
Using base 60 and encoding digits doesn't necessarily make the most sense.
For instance hours, it might make sense to use 2x times base 5, probably with no 0 encoded so that there's always something in the hours column.
Or, of course you could do an encoding with a base 4 and a base 6, which is mentally easy to convert back to to a regular 24 hour clock. TBH I'd probably encode that as maybe a square where there's 1-4 sides enclosing a polygon of 0-5 points.
Maybe I'd encode half-minutes with 3 base 5 encodings and some encodings unused, or a base 3, base 4 and a base 5 encoding for minutes.
I would probably also then group them into functional groups that touch each other within the group, so you can easily see the groups, but tell them apart easily.
Actually, I was going to completely edit this comment with a different suggestion, but decided to leave that to show the thought process.
Thinking more about the hours, I like the circle progressing round, but also maybe you could do it as a box with 1-4 sides present enclosing a polygon of 0-5 points. It's easy to mentally convert that to both 12 and 24 hour clock format.
For the minutes, I'd be tempted to do something like the current hours, done as a fraction of the circle arc with an additional polygon of 0-5 points for the minute. It's relatively easy to eyeball the size of the 5 minute segment, and the polygon just refines that a bit. You could locate the polygon so it touches the arc at the 1/12 of the circle corresponding to the base, which would help with estimating the arc length too.
Either way, I wouldn't make the encoding rely on colours, especially colours that look very similar.