Can not create serial number: Serial number is too long

As image, I can’t add a Custom Name that it’s length 255 to the license.

What RSA key length do you use?

2048 bits

I think I’ve figured out that a 2048-bit key can only encrypt up to 256 bytes of data.

After subtracting 11 bytes of nMinPadding, the remaining 245 bytes are available.

2048 bits = 256 bytes - this is the maximum size of a serial number and in your case it isn’t enought to store all required information (include the value of the “Name” field)