I have a question about dimensionality reduction. I want to understand how methods like MDS and t-SNE work. In particular, I'd like to understand the difference when I precompute the distance matrix or not.
To try to understand it, I took 10 images from the MNIST dataset, and created 10 copies of each one of them. Then I created a distance matrix with the Modified Hausdorff Distance between all the images. Finally, I used the TSNE and MDS functions available in sklearn.manifold to project them into a 2D space. I ran both functions twice, once with the precomputed distance matrix specified (metric='precomputed') and once with the default value.
So, I have 10 images that are repeated 10 times each. I would expect to see 10 points (one per different MNIST image) composed of 10 points (10 repetitions of each image) overlapping each other in both t-SNE and MDS spaces because the distances are repeated. I only see this when I don't specify the metric as 'precomputed', not in the other case where I see dispersion.
Why is this happening? I don't know if there's any article talking about this. I would very much appreciate it if you can help me with this question.