Here are some experiments on more complext data and with more layers: https://arxiv.org/abs/1712.03480

Unfortunately, it does not look very promising so far.