Salaams brother... you have asked a very broad question I hope to be as succinct as possible inshAllah.
Allah (swt) says in the Qur'an:
[33:22] Verily you have in the Prophet of Allah an excellent model, for him who fears Allah and the Last Day and who remembers Allah much.
[59:8] And whatsoever the Messenger gives you, take it; and whatsoever he forbids you, abstain from that. And fear Allah; surely, Allah is Severe in retribution
Both these ayats along with many others and our Shahadah itself makes it abundantly clear that we take whatever Muhammad (saw) gave us IE we take his (saw) sunnah. Where there may seem to be a contradiction or discrepancy between a saying of the prophet (saw) or an action I will defer the specifics to a scholar. But on principal we know that everything that Muhammad (saw) said or did was wahy IE revelation and not from himself (saw). Allah (swt) confirms this by saying, “[53:4] Nor does he speak out of his own desire.”
With this as our premise, you have asked, “why did the prophet (saw) prohibit the writing of his (saw) hadith?” The answer is quiet simple; it was so that his (saw) teachings are not mixed in with ayat’s of the Qur’an. We know that as revelation was sent down scribes like Zayd ibn Thabit (ra) wrote them down. These ayats were written down on all manner of things IE leaves, skin, etc. From this they were learned, studied and memorised. Suppose hadith’s were being written down, learned, studied and memorised at the same time as the Qur’an, the possibility of the mixing of Hadith’s with ayats of the Qur’an is there and so doing the aforementioned in relation to Hadith was forbidden. This in itself acts as 1 of many proofs that the Qur’an was not authored by Muhammad (saw) and instead is the work of Allah (swt).
You further asked, “why were the Hadith compiled 150-200 years after Muhammad (saw) died?” Again this is a simple issue if one was to look at what has happening at the time. Under the Khalifah Umar Ibn Al Khattab (ra) it was agreed by consensus of the sahabah’s (ra) to compile the Qur’an which was completed under the Khilafah of Uthman ibn Affan (ra). During the period of the Khilafah Rashida there was great expansion. Amongst the mujaheed were prominent sahabah’s who would settle in the conquered lands and teach the people. However 150 years into the Khilafah there were no sahabah’s and the state had expanded from Spain to India. Furthermore these people were not Arabs. Therefore compiling and collecting Hadith and distributing them in written form, like the Qur’an had been compiled and duplicated over 100 years earlier, meant the Islamic scriptures can be distributed and were accessible to all. Suppose there was no compilation of the Qur’an or the Hadith how were people in territories which were tens of thousands of miles away from the originating lands of Islam to access them? Furthermore they weren’t even Arabs who had no knowledge or access to the Arabic language. The sahabah’s themselves and the early generations IE tabieen did not need written compilations since they were in the presence of and taught by Muhammad (saw) himself. But 150 years after these generations of people and with the new reality of an ever expanding and cosmopolitan Khilafah compilation became a necessity so that our scriptures become accessible to all. Alhumdulillah.
Your brother in Islam