Made Pointer::BufferNumber a final field of the PointerImpl implementation since there can't be a non MemoryBuffer pointer anymore. This has an advantage in the memory visibility of the field when cleaning short living pointers.