Prior to the first NHL expansion in 1967, hockey in the Southern California area, had been present in one form or another, but never really successful. Because of its proximity to San Francisco, San Jose's exposure to hockey was due largely to the teams based out of the Oakland-San Francisco Bay Area. The California Hockey League had been formed in 1928 and brought in players from Canada, but by 1930 the league had to dissolve because of poor financials, which really means there was nobody coming to the games.

With the Pacific Coast Hockey League turning pro in 1948, ex NHL'er Eddie Shore brought the Oakland Oaks to the area, but by 1950, the league was in trouble and folded. The San Francisco Seals, a former pro team from Edmonton, Alberta joined the Western Hockey League in 1961. By 1964 they had won 2 consecutive titles and then in 1966 they moved to the Oakland Coliseum Arena for their final year in the WHL.

In 1967, the Oakland Seals were admitted into the NHL as part of the NHL's first expansion since 1942. Poor attendance, poor on ice results led to the franchise becoming the Cleveland Barons and moving to that city for the 1976-77 season.

In May of 1990, the owners of the Minnesota North Stars, George and Gordon Gund were allowed to sell their franchise and it was later moved to Dallas, Texas, and in return they were granted the rights for an expansion club in the Bay area that would enter the league in 1991-92. The hope was now that the region had expanded its population and was one of the technical centers in the U.S., there would be the support for another NHL hockey club.

The name of the team was selected through a worldwide contest. Entries came in from across the U. S. and Canada and as far away as Italy. The name was selected from over 5,700 entries and 2,300 different names. After selecting the name in September of 1990, George Gund announced that the team would base themselves in San Jose. The team would play their first two years in the Cow Palace in Daly City and move to a new arena in San Jose in 1993-94.

In the Expansion and Dispersal Draft on May 30, 1991, the Sharks obtained players like Arturs Irbe, Neil Wilkinson, Brian Hayward, Rob Zettler and Doug Zmolek, and later they selected several other players who were on NHL teams unprotected lists, the most notable was Jeff Hackett. On June 22, 1991 the Sharks selected Pat Falloon, Ray Whitney and Sandis Ozolinsh in the Entry Draft, and just before training camp began, general manager Jack Ferreira traded a minor league player and a draft pick for Doug Wilson, a 14 year NHL veteran who became the Sharks' first captain.

The Sharks began their history on October 4, 1991 with a 4-3 loss against the Vancouver Canucks. Craig Coxe scored the Sharks first ever goal and 4 days later at the Cow Palace, the Sharks posted their first win against the Calgary Flames with goalie Brian Hayward in net. Their first season was a 6th place finish in the Smythe Division on a 17-58-5 record. After the first season, gm Jack Ferreira was fired and replaced with what became known as the "San Jose 3 Headed Monster." Chuck Grillo, George Kingston and Dean Lombardi each had a 1/3 responsibility for managing the team, which would lead to eventual problem in running the time.

The Sharks second season wasn't much better. In the off season they had not acquired anyone of substance in the draft. Although they won their first game, they went on the skids and lost the next 9 games, and beginning Jan 4, 1992 they began a record setting 17 game losing streak (still and NHL record shared with Washington). Their worst loss in history came against Calgary, losing 13-1. But there was not all gloom and doom for the Sharks that year. Arturs Irbe posted the teams first shutout win on November 17, 1992 and Rob Gaudreau posted the teams first hat trick in just his second NHL game on December 3, 1992. He would do it all over again 4 games later against Quebec. Despite those couple of bright spots, the Sharks went on to set yet another NHL for the most losses in a single season with an 11-71-2 record, finishing in 6th place in the Smythe Division.

The Sharks began their third season in their new arena, the San Jose Arena with Kevin Constantine behind the bench. Despite their initial 0-8-1 start the Sharks rallied and finished the season with a 33-35-16 record, again setting another NHL record for the biggest single season turnaround (58 points) and finshed in 3rd place, earning their first post season berth. The Sharks faced first place Detroit in the first round and in a stunning upset eliminated the Red Wings in the final game of the series. In the second series they faced the Toronto Maple Leafs. In game 7, it appeared as though the Sharks were poised to edge Toronto as Sandis Ozolinsh sat poised to put the winner in, when he passed off the puck to Igor Larionov. The tide turned quickly for the Sharks and Mike Gartner scooped up the loose puck and skated down the ice to put in the game winner.

In the following lockout shortened season the Sharks again made it to the post season, albeit in the final game of the season. In the first round they faced the Calgary Flames eliminating them in game seven. In the second round the Detroit Red Wings appeared, and not wanting to have a repeat of the previous season, the Wings dominated the Sharks in a 4 game sweep.

The 1995-96 season was not as kind to the Sharks. In the off season Irbe was attacked by his dog and he required hand surgery, and he and back up goalie Wade Flahtley had terrible starts to the season, Larionov wanted to be traded and Sandis Ozolinsh was traded for goal scorer Owen Nolan. By the end of the season Constantine was replaced with Jim Wiley behind the bench, Irbe was traded for Chris Terreri, and Chuck Grillo was fired from the head office.

Dean Lombardi was faced with rebuilding the team. With all 41 games sold out for the 1996-97 season, he had some money to work with. He reengineered the team by securing players such as Todd Gill, Al Iafrate, Kelly Hrudey, Marty McSorley and Bernie Nichols. He also managed to land Chicago goalie Ed Belfour who would be an unrestricted free agent at the end of the season. But with Al Simms behind the bench, the players were not motivated to win and finished the season in 7th place in the Pacific Division, 19 points out of a playoff spot. That same season, Tony Ganato was awarded the Bill Masteron Trophy for perseverance and dedication to the game. Granato had undergone brain stem surgery and had made a successful comeback. By the end of that season, Belfour went to Dallas and was replaced by Mike Vernon, Simms was replaced by Darryl Sutter. New Sharks who also joined the team were Patrick Marleau, Murray Craven, Shawn Burr and Marco Sturm.

The 1997-98 season saw the Sharks return to winning form. Led by Jeff Freisen, Owen Nolan and Patrick Marleau, San Jose finished the season in 4th place in the Pacific Division and back in the playoffs after a 2 year drought. They had 34 wins to their credit, a team record at the time. In the first round of the playoffs they faced the Dallas Stars and were eliminated in 7 games.

In the 1998-99 season Owen Nolan was selected as the 6th Sharks captain. The Sharks continued their winning ways, again finishing 4th in the Pacific Division only to lose to the Colorado Avalanche in the Conference Quarter Finals. Owen Nolan led the charge into the 1999-2000 season. Supported by newcomer Vincent Damphousse as well as Jeff Freisen, Mike Ricci and Patrick Marleau the Sharks posted their best season ever with 35 wins and 87 points, again finishing 4th in the Pacific Division. They defeated the St Louis Blues in 7 games for the Conference Quarter Finals, and then faced the Dallas Stars, losing in 5 games.