Remembering Al Smith’s Restaurant

The sign and exterior of Al Smith’s Harbor Sights Restaurant – now the Ephraim Visitor Center – which operated under several names during its 50 years in business. Photo courtesy of Peter Smith.

One of the benefits of growing older is being able to remember people, places and events that younger generations don’t know. This fall will mark 50 years since I first set foot in Door County, when my family took the opportunity of a long teacher-convention weekend to come to Ephraim.

Soon we were coming to Ephraim multiple times a year, including long stays in the summer when my father had extended vacations from Beloit College or commuted back and forth to extend the summer stays even longer. 

By 1975, my parents had bought a home in Ephraim at the end of Spruce Lane, one block off Highway 42 (Water Street), and had moved to Door County full time. (I started college that fall.) My parents’ house was also one block from Al Smith’s Harbor Sights Restaurant, which operated during the tourist season.

Although it may seem difficult to believe, the building that is now the home of the Ephraim Visitor Center was once a restaurant and was originally built to house an ice cream parlor and gas station. Across the street from the restaurant, in a spot that’s now a village parking lot, was a sweep of grass at the back of which, sheltered beneath a canopy of trees, was the building that Al and his wife, Olive, called home. It was an odd building for a home, with a garage door dominating the first floor.

Al was an interesting fellow. Given that we’re talking about the past, I think he could best be described with the old-fashioned term “cantankerous.” Simply put, Al didn’t seem to like very many people. Likewise, not many people seemed to like Al.

I knew Paul Burton had recorded interviews with many of Ephraim’s “old-timers” years ago, so I asked him whether he had interviewed Al. Paul said he hadn’t and then added, by way of explanation, “Too prickly.”

One person Al did like, however, was my little sister Alison. Al often sat outside his house so he could take in the world without having to interact with many people. When Alison got off the bus and started her walk home up Spruce Lane, Al would often call her over, and they would sit and chat. She even persuaded a reluctant Al to let her interview him for a school project. 

Why Al took a liking to my sister remains a mystery, though the fact that she was often called Al rather than Alison may have contributed. My sister, however, said her experience led her to believe that older people — both men and women — just seem to be attracted to young people with curly red hair … and she would know.

I got along fine with Al, but mostly we just exchanged passing pleasantries. I ate at his restaurant a handful of times, and my recollection of those meals is homemade American food — nothing overly fancy, but very good. There was a small counter area with tables and chairs, and you could see into the kitchen via a pass-through area for food. Mostly I remember the spectacular view of Eagle Harbor through the oversized windows at the back of the building.

So, with limited knowledge of Al’s story, I needed help. I contacted my neighbor Emily Wright at the Ephraim Historical Foundation and my friend Charlie Calkins, the Badger Bibliophile, who collects all manner of Door County ephemera. Charlie supplied me with a sign advertising Al Smith’s Garage Paint and Body Shop in Sister Bay. Emily provided me with some background on the building itself, but she was unable to locate anything in the Foundation’s collection pertaining to the restaurant. 

In my search for photos of the restaurant, I asked a coworker to post my quest on Facebook. Within six hours, I was emailing back and forth with Al Smith’s grandson Peter Smith, who graciously sent me a host of photos and materials that help tell the story.

The building that now houses the Ephraim Visitor Center was built in 1929 by Milton Hanson for his daughters Olive (Al’s future wife) and Martha, and it was originally an ice cream parlor and gas station. When Al married Olive, he moved from Sister Bay to Ephraim, and, presumably, it was around this time that his name began appearing in the name of the restaurant. In addition to being called Al Smith’s Harbor Sights Restaurant, at one point it was also named Smith’s Luncheonette, and it may have had other names through the years as well.

The story behind the Al Smith’s Garage sign turned out to be interesting. Al’s garage was located on the property that now houses CHOP restaurant and the Dörr Hotel, now under construction. In 1945, Al sold the property to E.F. Anderson, who opened the Masterfreeze Home Locker Manufacturing Company. It was already a successful business in Milwaukee, so Anderson decided to open a satellite manufacturing operation in his hometown of Sister Bay. Originally located in Al Smith’s old body shop, Anderson put additions on the original building, and by 1948, he announced a production goal of 2,500 eight-cubic-foot electric home freezers for residential customers.

According to Peter Smith, Al was an accomplished welder who had worked in the shipyards during his early life. After Al married Olive and moved to Ephraim in 1945, Peter believes that Al may have continued to do some work for Masterfreeze welding cabinets. And Al’s welding work probably explains the unusual building that served as his home in Ephraim.

Peter noted that Al never worked in the restaurant that bore his name. Olive did all the cooking and hiring and ran almost every aspect of the business. This may seem odd, but this was at a time when she would have introduced herself as Mrs. Al Smith. You’d have called her Olive only if you were given permission; otherwise, she was Mrs. Smith.

The last days of the restaurant were rather tragic. Sometime during Halloween night in 1979, a “trio of youngsters,” the Door County Advocate reported, used black “spray paint to print obscenities” on the exterior of the restaurant and Al’s car. He was 77 at the time, living with a pacemaker and battling cancer. 

The damage to the sandstone exterior was significant and was going to be difficult to remove. In typical Al fashion, the Advocate quoted him as saying, “… all I can say is it’s a lucky thing for those kids that the sheriff got them first. With him they’ll be safe; with me, things wouldn’t have gone so good.” The same report referred to the restaurant as a “50-year-old Ephraim landmark.”

The restaurant opened for its final two days in August of 1980 — in order for Al and Olive to maintain their restaurant license — but it would never again be a restaurant.

In talking with Peter, I mentioned that I had found an Advocate story noting that Al and his son (Peter’s father) had finished second in Ephraim’s sons-and-fathers shuffleboard league in 1949, and I asked him about his grandfather’s reputation for being “difficult.” 

“He could be pretty rough around the edges to other people,” Peter conceded, “but he was a wonderful grandfather who loved his family.”