Jason Gould was born in Roxbury, New York, the son of Mary More (1798–1841) and John Burr Gould (1792–1866). Jay Gould studied at local schools and the Hobart Academy. His principal was credited as getting him a job working as a bookkeeper for a blacksmith. A year later the blacksmith offered him half interest in the blacksmith shop, which he sold to his father during the early part of 1854. Gould devoted himself to private study, emphasizing surveying and mathematics.

In 1856, Gould entered another partnership with Charles Mortimer Leupp, a son-in-law of Gideon Lee, and one of the leading leather merchants in the United States at the time. Leupp and Gould was a successful partnership until the Panic of 1857. Leupp lost all his money, while Gould took advantage of the opportunity of the depreciation of property value and bought up former partnership properties for himself.

Gould and James Fisk became involved with Tammany Hall, the New York City political ring. They made Boss Tweed a director of the Erie Railroad, and Tweed, in return, arranged favorable legislation for them. Tweed and Gould became the subjects of political cartoons by Thomas Nast in 1869. In October 1871, when Tweed was held on $1 million bail, Gould was the chief bondsman.

In August 1869, Gould and Fisk began to buy gold in an attempt to corner the market, hoping that the increase in the price of gold would increase the price of wheat such that western farmers would sell, causing a great amount of shipping of bread stuffs eastward, increasing freight business for the Erie railroad. During this time, Gould used contacts with President Ulysses S. Grant's brother-in-law, Abel Corbin, to try to influence the president and his Secretary General Horace Porter. These speculations in gold culminated in the panic of Black Friday, on September 24, 1869, when the premium over face value on a gold Double Eagle fell from 62% to 35%. Gould made a nominal profit from this operation, but lost it in the subsequent lawsuits. The gold corner established Gould's reputation in the press as an all-powerful figure who could drive the market up and down at will.

After being forced out of the Erie Railroad, in his 40s Gould started to build up a system of railroads in the Midwest and West. Beginning in 1879, he gained control of four western railroads, including the Union Pacific, which completed part of the transcontinental railroad, and the Missouri Pacific Railroad. By 1880, he controlled 10,000 miles (16,000 km) of railway, about one-ninth of the length of rail in the United States at that time, and, by 1882, he had controlling interest in 15% of the country's tracks. Because the railroads were making enormous profits and had control of rate setting, his wealth increased dramatically. When Gould withdrew from management of the Union Pacific in 1883 amidst political controversy over its debts to the federal government, he realized a large profit for himself.

Gould obtained a controlling interest in the Western Union telegraph company, and, after 1881, in the elevated railways in New York City. From 1868-1888, he was connected with many of the largest railway financial operations in the United States.

During the Great Southwest Railroad Strike of 1886, he hired strikebreakers. According to labor unionists, he said at the time, "I can hire one-half of the working class to kill the other half."

Gould was a member of West Presbyterian Church at 31 West 42nd Street.