In 1907, Ullman moved into a large house at 2150 15th Avenue South and in 1908 he entered into semi-retirement. Throughout the ensuing years, until his death in 1924, Ullman continued to take an active interest in the life of his family, his city and the world, and pursued with renewed vigor his lifelong avocation as a poet. In 1918 at the age of 78, Ullman penned a poem entitled "Youth" which became a favorite of General Douglas MacArthur, who regularly quoted from it in speeches. A framed copy of the poem was prominently displayed in MacArthur’s Tokyo office during the post-World-War II administration of Japan. Through General MacArthur’s influence, the poem became familiar throughout Japan and served to inspire a generation of Japanese citizens and business leaders faced with rebuilding their war-ravaged country.

Reflecting the admiration of the Japanese people for Mr. Ullman and his inspiring poetry, a Japanese business executive working in Alabama visited Birmingham in 1992 and expressed the desire to see the erstwhile Ullman residence where the poet had spent the last 17 years of his life and where he had penned his most significant work. Mr. Kenji (Ken) Awakura, then First Vice President of The Japan-America Society of Alabama (JASA), finding that the house had fallen into disrepair, was inspired with a vision of what the house could become if restored and operated as a museum.

Mr. Awakura spearheaded a JASA-led fund raising effort in Japan and the United States, resulting in corporate and personal contributions which were used to purchase and restore the property. The property was presented to The UAB Educational Foundation by JASA in 1993, and the Ullman Museum, displaying materials, artifacts, and furniture donated by members of the Ullman family, officially opened on March 21, 1994.

Additional generous gifts to JASA by the Alabama Power Foundation and Mr. Wyatt Rushton Haskell, a prominent Birmingham attorney and Ullman devotee, were invested to ensure the ongoing maintenance and improvements of the museum.

The Samuel Ullman Museum is located at 2150 15th Avenue South, Birmingham, Alabama.