Olfactory epithelium plays a critical role in pheromone detection

Howard Hughes Medical Institute researchers have discovered that pheromones essential for mating behavior in mice are recognized by the nose and not by the vomeronasal system, as researchers had long suspected.

The new studies demonstrate that the main olfactory epithelium, which was presumed to be mostly involved with the sense of smell, plays a critical role in pheromone detection.

Howard Hughes Medical Institute investigator Catherine Dulac and colleagues Hayan Yoon, an HHMI predoctoral fellow, and Lynn W. Enquist published their findings in an immediate early publication on November 10, 2005, in the journal Cell. Yoon and Dulac are at Harvard University, and Enquist is at Princeton University. Related studies by HHMI investigator Linda B. Buck are published in the same issue.

The pheromone communication system, which is found in a wide range of mammals, involves detection of chemical odorants released by animals. Pheromones are chemicals that are involved in changing behavior or hormone secretion. According to most biology textbooks, detection of pheromones takes place in a specialized structure, called the vomeronasal organ (VNO). Although the VNO resides in the nasal cavity, the pheromone sensory system is distinct from the sense of smell, as are the chemical receptors involved. In animals possessing a pheromone sensory system -- including mice, dogs, cats, and elephants -- the system governs a range of genetically preprogrammed mating, social ranking, maternal, and territorial defense behaviors.

In their experiments, Dulac and her colleagues sought to determine whether sensory neurons in the main olfactory epithelium and the VNO were connected to neurons in the brain that synthesize luteinizing hormone releasing hormone (LHRH). LHRH controls the onset of puberty. In females it also stimulates ovulation and controls the estrus cycle. In males, the hormone controls gonad function, including spermatogenesis and testosterone production.

"One of the classical dogmas is that LHRH neurons receive inputs from the vomeronasal system. For example, textbooks on reproductive physiology say that pheromonal modulation of reproductive behavior depends on these connections," said Dulac. "However, our studies show that this idea is wrong."

Previous experiments designed to tease out the routes of neurons used techniques that depended heavily on dyes that could not label specific populations of neurons. These experiments suggested that neurons of the vomeronasal system are directly connected with LHRH neurons, and thus carry the modulatory effect of pheromonal cues on LHRH release. But because classical tracing techniques could not specifically identify the connection of neuronal subsets, they ended up tracing the routes of groups of undefined neuronal populations clustered around LHRH producing neurons, Dulac said.

In the experiments reported in Cell, Dulac and her colleagues used a genetically altered version of a pseudorabies virus that is a highly specific tracer, permitting the researchers to label neurons with a fluorescent protein marker. This technique relied on a conditional approach that caused the pseudorabies virus to replicate only when it was triggered by a specific genetic signal. The researchers also created a transgenic mouse in which neurons expressing LHRH provided that signal, making the viral tracer highly specific.

"So, whenever we saw a signal from the conditional virus, we knew we were seeing connections to neurons expressing LHRH and nothing else," said Dulac. As a control, the researchers also used a "non-conditional" strain of pseudorabies virus that infected all types of neurons.

The researchers chose pseudorabies virus for their studies because it could be introduced into a neuron and infect its way upstream along the network of axons that connect neurons. The virus could also jump the synapses between neurons, as it moved from one neuron to the next. As a result, the researchers could use the pseudorabies virus to obtain a detailed look at all of the neurons that were connected to each other.

"Our conditional tracing studies showed a major projection to LHRH neurons from the main olfactory epithelium and the structures that comprise the primary olfactory complex," said Dulac. "This major route was never suspected. In addition, we were quite surprised when we realized we could not identify any connections with the vomeronasal system, which led us to attempt to confirm the experiment on a wide range of mice," she said.

As a control, the researchers performed a separate set of experiments using a non-conditional strain of the pseudorabies virus. These studies confirmed that the virus was capable of infecting all nuclei comprising the vomeronasal system.

Next, the researchers postulated that one should be able to identify functional correlates of the anatomical circuit linking olfactory neurons and LHRH-producing neurons. They studied the sexual behavior of mutant mouse strains that lacked either a functional olfactory epithelium or vomeronasal organ. They also studied the behavior of normal mice in which the olfactory epithelium had been selectively destroyed by a drug.

They found that the male mice that lacked a functioning olfactory epithelium showed little interest in females, with greatly reduced investigatory or mounting behavior. By contrast, male mice lacking a vomeronasal organ were able to mate. The researchers also discovered that LHRH neurons in the olfactory-deficient male mice were no more activated by exposure to female urine -- which normally elicits sexual behavior -- than exposure to water. Future studies will aim at further characterizing the network of connections from the olfactory system to LHRH neurons in many brain structures, said Dulac. Also, the researchers plan to explore how olfactory and vomeronasal neurons work together in mice to control reproductive behavior.

Dulac said that extending the conclusions from their studies in mice to humans is "totally speculative, but extremely interesting. There have been many observations of behavior in humans that suggest we react to pheromones," she said. "For example, it is known that women who live in close quarters tend to have synchronized menstrual cycles. But it was also believed that humans would need a functional vomeronasal organ in order to sense pheromones -- and there is no evidence for such an organ in humans.

"But now that we have identified an olfactory circuit that plays a role in sexual behavior in rodents, this requires us to rethink how mammals detect pheromones. It is quite tempting to speculate that there might be such a circuit in humans," said Dulac. "We must also rethink the simplistic division of olfaction, in which the main olfactory system governs cognitively based behavior and the vomeronasal system governs hard-wired, pheromone-triggered behaviors."