Deciphering a language which has left behind only a limited number of very short texts is hard. There are lots of undeciphered ancient languages; for additional distraction, some of those scripts might turn out to be representing non-languages, say, heraldic or ornamental symbols. Successful decipherments generally start from deep scholarship combined with elements of luck, rather than from patient application of a well known but tedious method.
Reconstructing a language which didn't leave behind any actual texts (literature or spoken recordings) is even harder. Of course you can easily conduct make believe reconstruction based on a small pool of arbitrary selection of "evidence" taken from "derived" languages, but then you may have hard time convincing your peers that your reconstruction is inevitably correct at least in its basic tenets. In the best case, your eventual reconstructed language will have
- massive explanatory power
- some predictive power
Explanatory power helps you study similarities and correspondences between attested derived languages. Ideally, the reconstructed language will be by far the simplest explanation of why the attested languages worked out the way they did. There will be a pretty robust body of "knowledge" (theory) about the reconstructed language which will be backed by a wide consensus of linguists. However, if you let several experts translate the same text into the reconstructed language, they will provide you with vastly different translations - with even more variability and hesitation than would be typical of translations into an actual living language. The different experts will have some level of agreement, but also considerable level of disagreement about what's the simplest explanation of the origin of the language family being studied. They will posit a somewhat different starting point and somewhat different rules of evolution from the reconstructed language into actual, attested languages.
Let's not forget about predictive power, too. Sometimes texts in a previously unknown derived language are unearthed; sometimes they are deciphered. Sometimes such events throw considerable extra light on previous reconstructions of the proto-language. Example: The Hittite language, being much older than current languages of Europe and India, preserves some proto Indo European languages which we call "laryngeals" today; but those laryngeals were predicted (in the abstract) before Hittite was deciphered.
The same example with a timeline, to appreciate the typical slow pace of the language reconstruction business:
- 1879: Ferdinard de Saussure posits certain proto-Indo European "coefficients sonantiques", hypothetical sounds of the proto-language which were not directly preserved in any Indo-European language known at the time, but whose existence would allow alternative explanations of the proto-Indo European vowel system.
- 1902: Jørgen Alexander Knudtzon is the first to suggest that Hittite, one of very many undeciphered ancient languages written in cuneiform, might be a member of the Indo European family.
- 1917: Bedřich Hrozný deciphers enough of Hittite to be able to publish its grammar, confirming its Indo European affiliation
- 1927: Jerzy Kuryłowicz identifies a Hittite sound "ḫ" (cuneiform is a syllabic script, so the phonology of Hittite is very much a reconstructed matter, too) to correspond to one of Saussure's "coefficients sonantiques". Only at this point, Saussure's theory starts gaining truly wide acceptance.
- Today, it is typical to believe that proto-Indo European had 3 laryngeals of which Hittite preserved 2 (in certain contexts), those 2 merged into only 1 sound.
Reconstructing a language which didn't leave behind any actual texts from other languages none of which left behind any texts either is even harder. Proto-Indo European is actually already at this level of difficulty. It is normally reconstructed not directly from modern European and Indian languages, but rather from likewise hypothetical Proto-Germanic, Proto-Slavic, Proto-Indo Iranian, and so on. (In fact you can distinguish almost as many "intermediate languages" between proto-Indo European and today's languages as you want; but the further you go, the more arbitrary it becomes.)
Going back in time and piling up reconstructions upon reconstructions is like building a tower from the mud, except that you might not notice the point when your tower has already collapsed into a mere heap of mud unless you are very careful about explanatory and predictive power of your reconstruction. This is evaluated through comparison to other, totally incompatible alternative theories, of which there are generally plenty. At some point all speculation becomes unconvincing, difficult to support with facts, and often quite disconnected from mainstream theories about the successor languages.
There is no shortage of attempts of going ever further back in various directions, and the effort eventually becomes comparable to pursuing archaeology without the benefit of any excavations from the era being studied: highly speculative and divergent.
We are hitting a wall of fog when we go just a few thousands of years before the earliest written texts which we can read.
Purely linguistic methods seem to be hitting an entropy barrier at the moment. Any major leap further back will probably require entirely new methods, be they purely linguistic ones (such as identifying and leveraging language features that are very stable through the ages) or other ones.
Could excavations (of material culture, not of more texts) actually join forces with linguistics, to see much further back? It's not inconceivable. Common botanical, zoological or technical vocabulary inside a language family might hint at where the ancestors lived, what they hunted, how they lived.
What about the graves? Our anatomical dispositions for speech are changing on an entirely different timescale than that of language change. However, genomics might succeed where study of skull shapes didn't; the human faculty of language seems to be a tremendously more complex cognitive function than our articulatory organs are and it appears that we "grow" this capability rather than just "learn" it, and that's why I'm entertaining the purely hypothetical possibility of a potential rich genetic correlate worth studying. However, while this genetic correlate may perhaps be rich, it might not be undergoing fast enough evolution to ever connect to the so far comparably modest achievements of historical linguistics methodologically.