How Well Does Computer Translation Work?

From this page, you'll be able to connect directly with AltaVista's computer translation program "Babel Fish" and evaluate how well it truly works.

But before we do that and before this website expresses any further opinion about the quality of "machine translation" let's listen to what the promoters of this program have to say about its quality.

The following quote comes directly from the home page of Babel Fish:

"Why Can't I Understand the Translation?"

"Translating languages is a very complex task. The translator works best when the text you wish to translate uses proper grammar. Slang, misspelled words, poorly placed punctuation, etc. can all cause a page to be translated incorrectly."

"Can I Trust the Translation?"

"Machine translation produces reasonable results in many cases. But you should not rely on it. As in other areas of life, honesty pays. If you want to use a translated text always acknowledge the translator, and append or reference the original text. This will put the translation into the right context and will help you avoid embarrassing misunderstandings."

In other words, what they are saying rather clearly is that there are still some problems connected with "machine translation." It took many decades for the developers of a few such systems to finally admit even this much. And even today it isn't too hard to encounter computational linguists who are still convinced that language and translation are a "trivial task," one that can easily be cracked once they have finally "discovered the key," and there is no way of talking them out of this.

It is the frequently expressed opinion of this author�both in other articles on this website and in professional presentations to groups of linguists and translators�that the "key" or the "right method" will never be found. And that any key or method that will more or less "work" for certain kinds of texts will fail to work for others.

But that's enough editorializing...

So let's take a look at our little evaluation test...

As time goes on, we will enrich this page with many different kinds of text, so you can see what kind of results "Babel Fish" produces with legal briefs, medical studies, novels, plays, poetry, perhaps even some Shakespeare.

But just to be a bit fair and reasonable, let's start with the kind of text that the makers of these "MT" programs claim are absolutely ideal for this purpose: a straightforward business text, even one with a slight computer slant.

Take a look at the following paragraph and try to become familiar with it. You'll be seeing it in quite a few versions as we go along.

The Green Map System is a globally connected, locally adaptable framework for community sustainability. Green Maps chart the sites of environmental significance in urban places around the world. Each map is created locally in a unique way by mapmaking teams of all ages and backgrounds. Everyone uses our collaboratively designed Green Map icons to identify, promote, and link their home's ecological resources, along with the places that make the environment special. See all the Green Maps at our website, www.greenmap.org. Find out about joining this global effort to cultivate the interconnections between nature and culture through Green Maps.

This text has been chosen not only because it is written in simple, straightforward English but because it is accompanied by both a Spanish and a French version that are equally well written and straight forward in each of those two languages. Although they are themselves quite likely translations from the English, they are good translations, good enough to qualify as original texts in their own right.

We are about to submit the Spanish version of this same text to Babel Fish in order to see how well it translates it into English. And we will be able to judge this quite objectively because we have the original English text just a few mouse clicks above us.

At this point, our evaluation test will proceed on two tracks...

If you want to connect to Babel Fish and submit the Spanish version for yourself, click here.
If you would rather save time and see the results right away, simply go on reading this page.

When we submit the Spanish equivalent of this paragraph to Babel Fish and ask it to translate the text into English, here is precisely what it produces (in other words we are not responsible for any of the mistakes):

The System of Green Maps is a frame global, and at the same time modifiable on local scale, to obtain sustentable communities. The green Maps show environmental sites of importance in the cities around the world. Each map is locally created in unique form by groups of all the ages and experiences. Each one of them, uses the icons of the Green Map to identify, to promote, and to conectar the local ecological resources, along with the places that make of the atmosphere a special place. Visit the page Web: www.greenmap.org where it will be able to see the variety of created maps and like participating in this colaborativo effort of interconectar nature and culture through the Green Maps.

Now let's look at them side by side:

English Original:

The Green Map System is a globally connected, locally adaptable framework for community sustainability. Green Maps chart the sites of environmental significance in urban places around the world. Each map is created locally in a unique way by mapmaking teams of all ages and backgrounds. Everyone uses our collaborativly designed Green Map icons to identify, promote, and link their home's ecological resources, along with the places that make the environment special. See all the Green Maps at our website, www.greenmap.org. Find out about joining this global effort to cultivate the interconnections between nature and culture through Green Maps.

Computer Back Translation from Spanish:

The System of Green Maps is a frame global, and at the same time modifiable on local scale, to obtain sustentables communities. The green Maps show environmental sites of importance in the cities around the world. Each map is locally created in unique form by groups of all the ages and experiences. Each one of them, uses the icons of the Green Map to identify, to promote, and to connectar the local ecological resources, along with the places that make of the atmosphere a special place. Visit the page Web: www.greenmap.org where it will be able to see the variety of created maps and like participating in this colaborative effort of interconectar nature and culture through the Green Maps.

Since some viewers have already read this text once in its correct form, they may just imagine that it sounds close enough to the original and conveys the meaning well enough. But this is because they are already familiar with the meaning, a kind of "persistence of memory" illusion�try reading this text to others and see how much sense they will be able to make of it.

But for most readers, the problems of this version will be only too obvious:

it contains actual errors, for instance "atmosphere" for "environment" and "frame global" for "global framework;"
where it cannot translate a word, it simply throws in that word in Spanish, for instance "sustentables" and "interconectar;"
the full meaning is simply not clear;
and, of course, it is not even remotely written in correct English.

Naturally, it is for the reader to decide how disturbing such an error level may be. Reading a brief paragraph in this style may not be particularly painful, but what would it be like if one were forced to read an entire technical manual written in this manner? (And yes, it would be worse than reading technical manuals as they are currently written!)

At this point there is no need for the author to intrude his judgment in any way. The whole object of this exercise all along has been a simple one: Let Readers Decide for Themselves!

And as a form of "lagniappe" (here that word is especially appropriate), let's take a look at what happens when the French version of this same text is translated into English. It's a bit more comprehensible, but all the problems noted above are still very much in evidence.

But once again, let readers decide for themselves. Here it comes now (with the same disclaimer about errors)...

While being open on the world, the green system of cartography is a framework of work locally adaptable for a durable development of the communities. The green cards identify the significant environmental sites located inside an urban framework. Each card is created locally and in a single way by multi-field teams of all ages. By using the collective icons, one identifies, causes and binds the ecological resources of the medium and the places which make our environment remarkable. Look at each green card on our Web site, www.greenmap.org. Initiate you with the green cartographic system and the manner of taking part in this collective effort in order to discover the link between nature and the culture.

Now let's put these two side by side:

English Original:

The Green Map System is a globally connected, locally adaptable framework for community sustainability. Green Maps chart the sites of environmental significance in urban places around the world. Each map is created locally in a unique way by mapmaking teams of all ages and backgrounds. Everyone uses our collaboratively designed Green Map icons to identify, promote, and link their home's ecological resources, along with the places that make the environment special. See all the Green Maps at our website, www.greenmap.org. Find out about joining this global effort to cultivate the interconnections between nature and culture through Green Maps.

Computer Back Translation from Spanish:

While being open on the world, the green system of cartography is a framework of work locally adaptable for a durable development of the communities. The green cards identify the significant environmental sites located inside an urban framework. Each card is created locally and in a single way by multi-field teams of all ages. By using the collective icons, one identifies, causes and binds the ecological resources of the medium and the places which make our environment remarkable. Look at each green card on our Web site, www.greenmap.org. Initiate you with the green cartographic system and the manner of taking part in this collective effort in order to discover the link between nature and the culture.

The overall structure of this "translation" is better than the one from Spanish, but at the considerable cost of translating "maps" as "cards"�thus making the piece potentially incomprehensible�and merging the first two sentences so that neither of them truly comes through. Plus which, "unique" becomes "single," "promotes" becomes "causes," and "find out about joining" becomes "Initiate you with..."

If you now want to switch to the active track and submit these or any other texts to BabelFish, you can start to do so by clicking here.

Or if you simply want to satisfy your curiosity about what the Spanish and French texts look like, here they are.

First the Spanish text:

El Sistema de Mapas Verdes es un marco global, y al mismo tiempo modificable a escala local, para lograr comunidades sustentables. Los Mapas verdes muestran sitios ambientales de importancia en las ciudades alrededor del mundo. Cada mapa es creado localmente en forma �nica por grupos de todas las edades y experiencias. Cada uno de ellos, usan los iconos del Mapa Verde para identificar, promover, y conectar los recursos ecol�gicos locales, junto con los lugares que hacen del ambiente un lugar especial. Visite la p�gina web: www.greenmap.org donde podr� ver la variedad de mapas creados y de como participar en este esfuerzo colaborativo de interconectar naturaleza y cultura a trav�s de los Mapas Verdes.

And here the French text as well:

Tout en �tant ouvert sur le monde, le syst�me de cartographie vert est un cadre de travail localement adaptable pour un d�veloppement durable des communaut�s. Les cartes vertes identifient les sites environnementaux significatifs situ�s � l'int�rieur d'un cadre urbain. Chaque carte se cr�e localement et de fa�on unique par des �quipes pluridisciplinaires de tous �ges. En utilisant les ic�nes collectives, on identifie, suscite et lie les ressources �cologiques du milieu et des endroits qui rendent notre environnement remarquable. Regardez chacune des cartes vertes sur notre site web, www.greenmap.org. Initiez-vous au syst�me cartographique vert et � la mani�re de participer � cet effort collectif afin de d�couvrir le lien entre la nature et la culture.

Free shipping on equipment purchases over $100