Maniphest T105304

upload.wikimedia.org down
Closed, ResolvedPublic
Actions

Assigned To

None

Authored By

	Steinsplitter
	Jul 9 2015, 10:22 AM

Tags

Referenced Files

None

Subscribers

View All 9 Subscribers

Description

Fileservers are down. All wikis affected.

Related Objects

Duplicates Merged Here: T105306: Images have stopped working on Commons

Event Timeline

Steinsplitter created this task.Jul 9 2015, 10:22 AM

Steinsplitter raised the priority of this task from to Unbreak Now!.

Steinsplitter updated the task description. (Show Details)

Steinsplitter added a project: acl*sre-team.

Steinsplitter subscribed.

Restricted Application added subscribers: Matanya, Aklapper. · View Herald TranscriptJul 9 2015, 10:22 AM

upload-lb.esams.wikimedia.org is DOWN: PING CRITICAL - Packet loss = 100%

and http://ganglia.wikimedia.org/latest/graph.php?r=hour&z=xlarge&h=vl100-eth0.lvs3002.esams.wmnet&m=bytes_in&s=by+name&mc=2&g=network_report&c=LVS+loadbalancers+esams

Ops are aware and working on it.

Matanya added a project: User-notice.Jul 9 2015, 10:33 AM

Matanya set Security to None.

We think we solved the issue, but we temporarily set the esams DC offline for upload while we investigate the problem further.

The public-facing issue should be solved since 10:38 UTC

Joe closed this task as Resolved.Jul 9 2015, 10:58 AM

Aklapper merged a task: T105306: Images have stopped working on Commons.Jul 9 2015, 12:38 PM

Aklapper added subscribers: Nemo_bis, Ricordisamoa, Krinkle and 2 others.

• gpaumier moved this task from To Triage to Announce in next Tech/News on the User-notice board.Jul 10 2015, 10:24 PM

• gpaumier moved this task from Announce in next Tech/News to Recently announced in Tech/News on the User-notice board.Jul 10 2015, 10:41 PM

Nemo_bis added a project: Wikimedia-Media-storage.Jul 11 2015, 7:16 AM

Restricted Application added a project: Multimedia. · View Herald TranscriptJul 11 2015, 7:16 AM

• gpaumier moved this task from Recently announced in Tech/News to Already announced/Archive on the User-notice board.Jul 16 2015, 6:42 PM

Maintenance_bot edited projects, added User-notice-archive; removed User-notice.Aug 13 2022, 1:07 PM