Problem

FFT encoding Lyrics and the quantized f0 melody produces [batch size, hidden channel, time(seq_len) -> sequence length]. The sequence lengths can be different, right??

Solution

Use mean pooling~~

Take the mean for each feature in the hidden channel to make the seq len 1 -> do the same for the melody

ex.)

  ------hidden channel---------
김  
민
재
짜
스
  mean mean mean mean mean mean ....