The Hong Kong Rugby Football Union (now Hong Kong Rugby Union) was formed in 1952 and was affiliated to the Rugby Football Union (RFU) at Twickenham. Refereeing matters were originally handled by W C Richardson and Dr Erbele in the early 1950's.

The Hong Kong Society of Rugby Football Union Referees (HKSRFUR) was formed in 1957 and adopted a logo comprising a silver dragon on a green background with the letters 'HKSRFUR' underneath. Some years later, after the departure of Richardson and Erbele, the logo was changed to a dragon rampant rather than a sleeping one and to gold rather than silver.

In 2020 the Society renamed itself to the Hong Kong Rugby Referees Society. Today the society has more than 100 active Referees and Coaches serving 17 rugby clubs comprising more than 57 senior men's teams, 24 senior women's teams, as well as youth, schools and mini rugby teams. Each season this involves over 1000 appointments to games within Hong Kong and additional appointments to overseas tournaments and fixtures.

The society, fully supported by HKRU’s Referee Department, organises training and development courses for Referees and Referee Coaches at all levels of men's, women's, youth and mini games.