No, k-means is sensitive to the initialisation stage where centroids are randomly assigned to positions in the data space.

Yes! Each centroid is updated to the average position of the datapoints which were assigned to it in the previous iteration. If the previous update in centroid position did not result in new datapoints being assigned to it then it's position will not be updated.

If you keep increasing the number of centroids, at some point K will equal the number of data points. This will result in each data instance being assigned its own unique cluster. You will be fitting the spurious noise, not the underling trend of the data! The challenge with k-means is picking the correct number of centroids for the problem.

As the centroids are simply updated to the average position of the assigned clusters there is no guarantee of convergence on a global optimum. Rather convergence on local minima subject to cluster initialisation occurs.

Assuming that there are no datapoints with identical attributes there will always be a positive mean distance between clusters and their assigned datapoints if a centroid has more than one data point assigned to it.

Check out the 'Elbow' method in the slides. The sharp plateauing of the decline score with increasing number of K suggests the point where you stop modelling the true underlying clusters of the data and start to model noise.

Check the slides!

Check definitions in slides

Responsibilities define the probability of each data point belonging to each cluster. Remember GMM-EM is a soft assignment!

k-means is a hard assignment where as GMM-EM is a soft assignment. Every point belongs to all clusters corresponding to the responsibility.