FP16 inference

[Don-Chad]

Awesome job so far!!

@remsky

The original kokoro does allow FP 16 loading, and it speeds up about 25% or so! Would be great to implement this. (I tried and the architecture does not allow it currently)

The converting to fp16 is very straight forward, probably can be done instantly (model.half) also.

from hashlib import sha256
from pathlib import Path
import torch

path = Path(file).parent.parent / 'kokoro-v0_19.pth'
assert path.exists(), f'No model pth found at {path}'

net = torch.load(path, map_location='cpu', weights_only=True)['net']
for a in net:
for b in net[a]:
net[a][b] = net[a][b].half()

torch.save(dict(net=net), 'kokoro-v0_19-half.pth')
with open('kokoro-v0_19-half.pth', 'rb') as rb:
h = sha256(rb.read()).hexdigest()

[remsky]

Using the half model is implemented in the v0.1.2-pre and up branch image, and the option for either should be available there as well

[Don-Chad]

fantastic! Thank you