CAFÉ culture has been at the heart of Western society for hundreds of years, and since the late 20th century, the coffee house has been a focal point in some of our most iconic films and TV series, from Friends to Twin Peaks, Amelie to When Harry Met Sally. Meanwhile, high street domination by café chains such as Costa and Starbucks shows no sign of abating.

Or does it? When Costa Coffee recently announced a 1.5 per cent dip in sales for the third quarter of 2017, commentators speculated as to whether "peak coffee" had been reached. And while UK coffee shop sales grew by 7.3 per cent last year according to industry monitor Allegra, Telegraph business writer Bradley Gerrard pointed out that "growth was down a third on 2016 and less than half the rate of 2015". In a straitened consumer climate, could a caffeine crash be in sight?

It seems unlikely. Not least, because the coffee house is one of our oldest, and most influential, institutions. Coffee's earliest roots can be traced to Sufi religious orders in Yemen (who used it in ceremonies – probably as a stimulant) and, before that, to the highlands of Ethiopia. From there, it spread across the Islamic world in the 15th and16th centuries, Cairo becoming the centre of the coffee trade and the birthplace of the coffee house, whose rise (and fall) would mirror that of coffee itself. The first coffee houses sprang up around Cario’s Azhar University, spreading to Syria then to the Ottoman Empire, with coffee houses opening in the capital Istanbul by the middle of the 16th century.

The coffee house marked coffee’s migration from religious orders into wider society. Since Islam forbade alcohol, there was no equivalent of the tavern and the coffee house filled a social gap, providing a public space where men of all strata of society could meet and socialise. In so doing, it transformed the way that society interacted, aiding the spread of news and ideas. This, in turn, led to several backlashes against the coffee house as the authorities tried to ban them, worried by the upheaval in the social order. These bans, however, were always short-lived and the march of coffee and the coffee house continued unabated.

From the Islamic world, coffee spread westwards into Europe, a journey that owed much to simple trade, although war and conquest played a part. An unlikely early runner in the race to claim to be home to Europe’s first coffee house is Malta. In 1565, the Great Siege of Malta was lifted and the invading Ottoman Empire was repelled. A number of the invaders were captured and held as slaves, where they continued their practice of making and drinking coffee.

It’s obvious that trade played a role here since it’s unlikely the captured soldiers had an infinite supply of coffee with them. Malta, in the centre of the Mediterranean, was well-placed to trade with the Ottoman Empire to the east, hostilities notwithstanding. The drinking of coffee spread into the Maltese high society and from there into the general population, with coffee houses opening across Malta.

Unsurprisingly Venice, another major trading nation with links to the eastern Mediterranean, has a strong claim to mainland Europe’s first coffee house – which opened in 1645 – although coffee itself had been traded at least 30 years before that, being used mainly for medicinal purposes. London was not far behind, with its first coffee house opened in St Michael's Alley in 1652 by Pasqua Rosée, the Greek servant of British merchant Daniel Edwards, who lived for a while in Smyrna on the Aegean coast of Anatolia. Pasqua reputedly brewed the best coffee in the Ottoman Empire, which was “black as hell, strong as death, sweet as love”, according to a well-known Turkish saying.

The coffee house was an immediate hit, with Pasqua serving 600 dishes of coffee a day within a few months of opening. In just 50 years, there were over 3,000 coffee houses in London alone. This pattern was repeated throughout Europe as coffee houses were established in large trading cities and by the end of the 17th century, the coffee house had become firmly established in European culture, just as it had in the previous century in Islamic society.

Although Christian Europe and the Islamic Ottoman Empire had many differences, their coffee house cultures were remarkably similar, taking as a model the coffee houses first established in Cairo. So what was special about these institutions?

An early 16th-century Arabic poem commonly entitled In Praise Of Coffee includes the line: “Coffee is our gold. Wherever it is served, one enjoys the society of the noblest and most generous men.”

This concept of companionship seems to be at the heart of the rise of the coffee house. As an old Turkish saying goes: “Not the coffee, nor the coffeehouse, is the longing of the soul. A friend is what the soul longs for; coffee is just the excuse.”

This phenomenon, of people coming together over coffee, is still with us today. When someone says “let’s go for coffee”, do they really mean coffee, or is it an excuse for a catch-up?

One of the coffee house’s main roles was in the exchange of information. Early illustrations of London coffee houses show long, communal tables, crowded with men (unlike today, no respectable woman would be found in a coffee house in 17th-century London). Conversation with strangers was positively encouraged. New customers would be greeted with the cry of, “What news have you?” or more formally: “Your servant, sir, what news from Tripoli?”

This was noted by Samuel Pepys, writing in his diary on October 19, 1663. Describing his first visit to a coffee house in Cornhill, he notes that there was "much talk about the proceedings of the Turks. We hear that the plague has got to Amsterdam, brought by a ship from Algiers; it has also reached Hamburg”.

Coffee houses' reputation as sources of knowledge was such that they became known as penny universities, reputedly because the price of admission was one penny (the cost of a bowl of coffee).

At first, the coffee house was a spectacular success. In London alone, there were between 1,000 and 8,000 at the start of the 18th century. In contrast, Amsterdam, the hub of the European coffee trade, could muster just 32. However, London’s status as Europe’s coffee capital didn’t last, the drink’s success being eclipsed by that of tea, driven in part by Princess Catherine of Braganza who married King Charles II in 1662. Tea was the new Queen’s drink of choice, which led to its adoption in high society. It may also have increased its appeal to women, who were largely excluded from coffee houses, leading to its spread to all classes of society.

Some argue it was the decline of the coffee house which prompted the increase in tea drinking, with the coffee house the source of its own downfall. Its exclusiveness, and specifically, its exclusion of women, made coffee something that wasn’t drunk at home. In 1674, the Women's Petition Against Coffee railed against "excessive use of that newfangled, abominable, heathenish liquor which ... has so eunucht our husbands ... that they are become as impotent as age".

It also warned that by "frequenting these Stygian tap-houses", men would "soon learn to exceed us in talkativeness".

By the early 1700s, the coffee house had lost its novelty and, some argue, snobbery led the once-famed “penny universities”, accessible to all, to become the enclave of the few. All this led to a decline in coffee-drinking in Great Britain, a state of affairs that was to last until the even more short-lived post-war espresso bar boom which (once again) was focused on London.

Post-war London was in many ways, a drab place. Food rationing only ended in 1954 and the city was rebuilding after the devastation of the war. Rents were cheap, disposable incomes were growing and there was a newly-emerging youth culture.

The breakthrough came with an itinerant Italian dental-equipment salesman, Pino Riservato. Appalled by the quality of the coffee (often made from chicory and coffee essence), Riservato opened the Moka Bar, London’s first espresso bar, in Soho in 1953. An immediate success, it was soon followed by a host of others. Unlike our modern craze for chains, these early espresso bars didn’t imitate the Moka Bar in either look or feel. Instead, a host of independent operators moved into the market, each catering to a particular clientele.

Often they had little experience in the catering trade and virtually no experience in coffee. However, the entry requirements were low. Stores (like the bomb-damaged laundrette that became the Moka Bar) were available at very low rents and, other than the cost of the espresso machine, they could be fitted out cheaply. What’s more, since they didn’t serve alcohol, an expensive licence wasn’t required, although most found that to make ends meet, they needed to stay open late into the evening and serve food rather than coffee.

The boom, however, was short-lived, petering out in the mid-1970s. It’s uncertain why espresso bars were unable to become a permanent feature of UK life, but it’s clear that from the mid-1960s onwards, they faced stiff-competition with nightclubs and dedicated music venues taking over.

By the time I really started to notice coffee, in the early 1980s, coffee in the UK largely meant instant coffee and coffee shops were practically non-existent. Back then, the height of coffee sophistication was filter coffee, usually made in a cheap, electric brewer, using pre-ground beans. That this has changed so dramatically and so quickly is partly down to a very specific American influence.

In 1971, Starbucks was founded in Seattle, although it wasn’t until 1983, when Howard Schultz, the newly-installed head of marketing, went to Milan and discovered Italian espresso bar culture that the modern Starbucks was born. Starbucks’ growth was spectacular, from six Seattle stores in 1987, to 55 in 1989, based in the Pacific Northwest and Chicago. By 1996, Starbucks had spread across America with over 1,000 stores.

Back in the UK, things were moving more slowly. However, that began to change with the arrival of Friends, a popular US sitcom in 1995. A large part of Friends centred around the Central Perk, a coffee shop in New York's West Village, where one of the characters worked as a waitress and where the others would gather. Friends made the coffee shop seem a welcoming place, a cool place, a place that people wanted to be.

Soon the coffee shop scene was taking off in the UK, with home-grown chains such as Costa opening branches around the country. Another major player was the Seattle Coffee Company, bought by Starbucks in 1998. Since then, the growth of the coffee shop chains has been spectacular in the UK. At the end of 2015, the market leader was Costa, with over 1,500 stores around the country. Starbucks, in comparison, had just over 700, with Caffè Nero coming in third with 500. Although the chains sell themselves with a firm nod towards Italian coffee culture, their actual cafés, in terms of layout, service and product, are firmly in the American model made wildly successful by Starbucks.

Starbucks coined the phrase “third place” to describe the coffee shop, somewhere beyond work and home where people could gather informally, much as they did in Friends. These days, people also use coffee shops as places to work, with the laptop (and increasingly the smartphone) generation using them as mobile offices. In part this has sustained the coffee shop boom through two major recessions, with growth only just showing signs of slowing down, as noted at the beginning.

So has the recent rise been just another boom, or something more permanent? The great fear is that we are living in a bubble, that in 20 years’ time, we’ll look back and think that this was a wonderful time for the coffee lover. While predicting the future has always been a foolish activity, I’m optimistic, although coffee faces some significant challenges, not least climate change and the need to provide sustainability for coffee farmers around the globe.

I think, however, that the coffee house culture is here to stay. Much like the coffee house of the 16th-century Islamic world and of London a 100 years later, it seems to be filling a need. Coffee remains an affordable luxury for many and coffee shops are inclusive spaces (a far cry from the men-only versions of 17th century London).You won’t find as much conversation in a modern coffee shop as in the old penny universities, but the spirit lives on. While people’s noses are buried in their laptops and smartphones, you’ll find they’re surfing the web or browsing social media, the modern-day equivalent to: “Your servant, sir, what news from Tripoli?”

The modern day boom isn’t just confined to London. I’d argue that some of the best and most innovative advances in coffee are happening across the UK. In particular Edinburgh and Glasgow have thriving coffee scenes, with some of the best coffee shops in the country and some high-class coffee roasters as well.

Despite the challenges and coffee’s boom-and-bust past, I have faith that the coffee shop is here to stay. Rest assured, as long as there are people in the world growing, roasting and serving excellent coffee, I’ll be out there, finding the best places to drink it and writing about them.

Brian Williams is author of The Philosophy Of Coffee, published by British Library, £9.99. To read his popular blog, The Coffee Spot, visit www.brian-coffee-spot.com