Hi @edomt ,

I have noticed that the data stopped being updated after Feb 14th for Serbia.

In order to help resolve the issue I have collected the list of data sources for pull requests that contain the news articles since the first mention number of the fully vaccinated persons.

If you need further help with translation/validation I will be happy to assist. Furthermore, I attempt to reach the government officials (official channels, social media, private contacts...) in order to ask them to provide automated feed, sadly no success yet.

Please study the entries and notes I have provided bellow.

PR | Publication date | Vaccinated | 2nd | 1st | Note | Link | Original sentence | English translation -- | -- | -- | -- | -- | -- | -- | -- | -- #448 | 07.02.2021 | 544209 | 0 | 544209 | The article mentiones that the second dose of the mass vaccination will start from 09.02.2021 | https://www.danas.rs/drustvo/vakcinu-protiv-korona-virusa-u-srbiji-do-sada-primilo-544-209-osoba/ | Vakcinu protiv korona virusa u Srbiji je do sada dobilo 544.209 osoba, a u utorak, 9. februara, kako je najavljeno, počinje i revakcinacija. | So far 544209 persons have received vaccine. On Tuesday Feb 9th, as announced, the revaccination starts. #468 | 10.02.2021 | 592634 | 47665 | 544969 |   | https://www.alo.rs/vesti/drustvo/592-634-primilo-vakcinu-sve-veci-broj-revakcinisanih/383807/vest | Prema najnovijim podacima, u Srbiji je do sada ukupno vakcinisano 592.634 ljudi, dok je 47.665 osoba primilo drugu dozu vakcine. | According to the lates data, a total of 592.634 people have been  vaccinated, among them 47.665 persons have received second dose of vaccine. #475 | 11.02.2021 | 636119 | 83262 | 552857 |   | https://www.kurir.rs/amp/3622973/danas-tacno-2000-novozarazenih-najnoviji-korona-presek-u-srbiji-15-preminulih-raste-broj-pacijenata-na-respiratorima | Dosad je ukupno vakcinisano 636.119 građana dok je drugu dozu primilo 83.262. | Until now 636.119 citizens have been vaccinated, among them 83.262 have received second dose. #486 | 13.02.2021 | 803319 | NO DATA | NO DATA | I could not locate the information on number of fully vaccinated for this data point. IMHO: it should be omitted | https://nova.rs/vesti/drustvo/u-srbiji-vakcinisano-803-319-ljudi/ | U Srbiji je, prema najnovijim podacima Vlade, 803.319 ljudi vakcinisano protiv koronavirusa. | According to the latest data provided by Government, 803.319 people habe been vaccinated against corona virus. #497 | 16.02.2021 | 944222 | 304767 | 639455 |   | https://www.redportal.rs/srbija/12168/korona-u-srbiji-skoro-1900-novozarazenih-16-preminulo | U Srbiji je ukupno vakcinisano 944.222, dok je revakcinisano 304.767 osoba. U Srbiji je ukupno vakcinisano 944.222 (osoba), dok je revakcinisano 304.767 osoba. | In Serbia total of 944.222 (persons) has been vaccinated, among them 304.767 persons have been revaccinated. Note: adjective in Serbien language has to be alligned with a noun both in term of multiplicity, gender and case.  Due to this fact a noun can be omitted if it is later mentioned in the same sentence with again compliant multiplicity, gender and case. In this case "osoba" (persons) was omitted from the first part of sentence. #497 | 17.02.2021 | 1002949 | 356010 | 646939 | For 17.02.2021, initial pull request contains incorrect data for total number of vaccination. Most likely the article got updated in the meanwhile. | https://www.rtv.rs/sr_lat/drustvo/u-srbiji-945.989-vakcinacija-druga-u-evropi-sedma-u-svetu_1209637.html | Do sada je vakcinisano 1.002.949 građana. | Up to now 1.002.949 citizens have been vaccinated #523 | 18.02.2021 | 1111000 | Impossible value: 694488 | #VALUE! | This data entry is invalid and should be removed. I tried my best to locate alternative data source. My assumption is that the health offical either switched 2nd and 1st dose or was misquoted by the journalist. All other articles seem to derive the text from the original, provided by the news agency FoNet/Beta. Note: I also tried switching the values, and it was consistant with prior and later numbers, but this data point can not be used. | https://www.danas.rs/drustvo/krizni-stab-vakcinisano-1-111-000-gradjana-drugu-dozu-primilo-skoro-695-000/ | Pokrajinski sekretar za zdravstvo Zoran Gojković izjavio je da je do danas protiv korona virusa u Srbiji vakcinisano 1.111.000 ljudi i naglasio da Krizni štab ne može „da ide i pojedično osuđuje za ponašanje koje nije u skladu sa epidemiloškim merama“. | Provincial secretary Zoran Gojkobvić has  announced that until today 1.111.000 people have been vaccinated and empasised that the Crisis Headquarter can not be held responsible for penalizing the individual diviations from the epidemic measures. #525 | 20.02.2021 | 1136131 | 418917 | 717214 | This one was not provided within pull request but rather as issue. I can confirm the translated figures | https://www.rts.rs/page/stories/sr/%D0%9A%D0%BE%D1%80%D0%BE%D0%BD%D0%B0%D0%B2%D0%B8%D1%80%D1%83%D1%81/story/3134/koronavirus-u-srbiji/4265914/korona-srbija-podaci-zarazeni.html | Prema najnovijim podacima Vlade, u Srbiji je dato 1.136.131 doza vakcina protiv kovida. | According to the latest data provided by government, 1.136.131 dosis of vaccines against covid have given. #544 | 23.02.2021 | 1263111 | 451658 | 811453 |   | https://www.b92.net/info/vesti/index.php?yyyy=2021&mm=02&dd=23&nav_category=12&nav_id=1816193 | U Srbiji je, prema najnovijim podacima Vlade, izvršeno 1.263.111 vakcinacija protiv koronavirusa, a 451.658 građana dobilo je drugu vakcinu. | In Serbia, according to the latest data form Government, 1.263.111 vaccination against corona virus has been executed and 451.658 citizens has received the second vaccine. #550 | 23.02.2021 | 1.275.844 | 462578 | 813.266 |   | https://nova.rs/vesti/drustvo/nova-rs-otkriva-koliko-je-ljudi-zaista-vakcinisano-u-srbiji/ | "Prvu dozu vakcine je primilo 813.266 građana, dok je drugu dozu primilo njih 462.578”, navode iz Kancelarije za IT i eUpravu za Nova.rs. Dalje, preciziraju da je u Srbiji, sa današnjim presekom, ukupno bilo 1.275.844 vakcinacija, od čega revakcinacija 462.578. | "First dose of vaccine has been received by 813.266 citizens, while the second dose of vaccine has been received by 462.578", quated from the "Office for IT and eGovernment" for the portal Nova.rs. Further, they precisely say that today with current state a total of 1.275.844 of vaccinations have been made, of which 462.578 2nd.

This is a centralized issue to discuss how to jointly handle two-dose and one-dose vaccines in the data.

As of 2021-01-12, all approved vaccinations are administered in two doses. However, some future vaccines are announced to require only one dose.

A first proposal could be as in England.csv, where a new column people_fully_vaccinated has been introduced. Therefore, when analyzing all doses jointly (from all providers), we might prefer to use this column instead of total_vaccinations.

2 Doses

Pfizer/BionTech, Moderna, AstraZeneca/Oxford university, Novavax, Medicago/GSK, CureVac, Gamaleya, Sinopharm I, Sinopharm II, SinoVac, Bharat Biotech

1 Dose

Johnson & Johnson, CanSino

Unclear

ZFSW

Reference: Covid vaccine tracker: The shots available and the doses administered, Financial Times, January 4 2021 (accessed on January 12, 2021)

As far as I know, there are currently three approaches to automate vaccination data updates:

1. Automatic updates
   • Incremental scraping: Country reports last numbers. These are retrieved and appended to existing country data
   • Batch importing: Country reports all historical data. In this case, the entire country's data is overwritten and updated daily.
2. Manual updates The data is manually updated using internal OWID tools. To contribute for manual updates, report new data in https://github.com/owid/covid-19-data/issues/230 or open a new issue.

UPDATE: There is now the file automation_state.csv which lists all countries with automated processes (thanks @edomt)

UPDATE: Read Contribute section for more details. ℹ️

Would be good to double-check against national health authority databases to confirm whether booster data are indeed NA as reported in OWID. It appears many countries have started boosting.

Currently OWID reports NA for 85 countries

DONE

• [x] Afghanistan: No boosters
• [x] Angola: Added in https://github.com/owid/covid-19-data/commit/0a6e98af98d0d37f40809c2c94dac1ee6d6e086b
• [x] Antigua and Barbuda: No Boosters
• [x] Barbados: No Boosters
• [x] Belize: Sourcing from PAHO - Added in 593e9c5eb8dcc71add243371caa7ba164b5b17a8
• [x] Benin: No boosters based on https://africacdc.org/covid-19-vaccination/
• [x] Bhutan: 2018021864ea87ee7b056b8295affa064b949a9e
• [x] Botswana: Added in https://github.com/owid/covid-19-data/commit/0a6e98af98d0d37f40809c2c94dac1ee6d6e086b
• [x] Burkina Faso: No boosters based on https://africacdc.org/covid-19-vaccination/
• [x] Burundi: No boosters based on https://africacdc.org/covid-19-vaccination/
• [x] Dominica: No boosters - PAHO
• [x] Cabo Verde: Added in https://github.com/owid/covid-19-data/commit/0a6e98af98d0d37f40809c2c94dac1ee6d6e086b
• [x] Cameroon: No boosters based on https://africacdc.org/covid-19-vaccination/
• [x] Central African Republic: No boosters based on https://africacdc.org/covid-19-vaccination/
• [x] Chad: No boosters based on https://africacdc.org/covid-19-vaccination/
• [x] Comoros: No boosters based on https://africacdc.org/covid-19-vaccination/
• [x] Congo (Brazzaville): No boosters based on https://africacdc.org/covid-19-vaccination/
• [x] Congo (Kinshasa): No boosters based on https://africacdc.org/covid-19-vaccination/
• [x] Cote d'Ivoire: No boosters based on https://africacdc.org/covid-19-vaccination/
• [x] Djibouti: No boosters based on https://africacdc.org/covid-19-vaccination/
• [x] Eswatini: No boosters based on https://africacdc.org/covid-19-vaccination
• [x] Gabon: No boosters based on https://africacdc.org/covid-19-vaccination/
• [x] Gambia: No boosters based on https://africacdc.org/covid-19-vaccination/
• [x] Egypt: Added in https://github.com/owid/covid-19-data/commit/0a6e98af98d0d37f40809c2c94dac1ee6d6e086b
• [x] Equatorial Guinea: No boosters reported - source
• [x] Eritrea: Has not started vaccination campaign - WHO source
• [x] Ethiopia: No boosters reported - source
• [x] Fiji: Boosters added #2513
• [x] Grenada: Sourcing from PAHO - Added in 593e9c5eb8dcc71add243371caa7ba164b5b17a8
• [x] Guinea: No boosters reported - source
• [x] Guinea-Bissau: No boosters based on https://africacdc.org/covid-19-vaccination/
• [x] Ghana: Added in https://github.com/owid/covid-19-data/commit/0a6e98af98d0d37f40809c2c94dac1ee6d6e086b
• [x] Haiti: No boosters - PAHO
• [x] Iraq: WHO to release this data in the upcoming weeks
• [x] Jordan: Boosters started but not reported. WHO to release this data in the upcoming weeks
• [x] Kazakhstan: Boosters added in aaf94c1
• [x] Kiribati: SPC to release this data in the upcoming weeks
• [x] Korea, North: Has not started vaccination campaign = source
• [x] Kyrgyzstan: No booster data available (https://vc.emed.gov.kg/)
• [x] Laos: No Boosters. Might start campaign soon [ref].
• [x] Lesotho: No boosters based on https://africacdc.org/covid-19-vaccination/
• [x] Liberia: No boosters based on https://africacdc.org/covid-19-vaccination/
• [x] Libya: Added in https://github.com/owid/covid-19-data/commit/0a6e98af98d0d37f40809c2c94dac1ee6d6e086b
• [x] Madagascar: No boosters based on https://africacdc.org/covid-19-vaccination/
• [x] Malawi: No boosters reported - source
• [x] Mali: No boosters based on https://africacdc.org/covid-19-vaccination/
• [x] Mauritius: Added in https://github.com/owid/covid-19-data/commit/0a6e98af98d0d37f40809c2c94dac1ee6d6e086b
• [x] Mauritania Added in https://github.com/owid/covid-19-data/commit/0a6e98af98d0d37f40809c2c94dac1ee6d6e086b
• [x] Mexico: No boosters - official report, PAHO
• [x] Morocco Added in https://github.com/owid/covid-19-data/commit/0a6e98af98d0d37f40809c2c94dac1ee6d6e086b
• [x] Mozambique Added in https://github.com/owid/covid-19-data/commit/0a6e98af98d0d37f40809c2c94dac1ee6d6e086b
• [x] Myanmar: No boosters reported
• [x] Namibia: No boosters based on https://africacdc.org/covid-19-vaccination/
• [x] Nauru: SPC to release this data in the upcoming weeks
• [x] Nepal: Boosters added in #2483
• [x] Nicaragua: No boosters - PAHO
• [x] Niger: No boosters based on https://africacdc.org/covid-19-vaccination/
• [x] Nigeria: No boosters based on https://africacdc.org/covid-19-vaccination/
• [x] Papua New Guinea: SPC to release this data in the upcoming weeks
• [x] Romania: Added boosters in caedfcf, asked source maintaners for booster data.
• [x] Sao Tome and Principe: Added in https://github.com/owid/covid-19-data/commit/0a6e98af98d0d37f40809c2c94dac1ee6d6e086b
• [x] Saint Lucia: Boosters from (https://ais.paho.org/imm/IM_DosisAdmin-Vacunacion.asp)
• [x] Samoa: SPC to release this data in the upcoming weeks
• [x] Seychelles: Added in https://github.com/owid/covid-19-data/commit/0a6e98af98d0d37f40809c2c94dac1ee6d6e086b
• [x] Senegal: No boosters based on https://africacdc.org/covid-19-vaccination/
• [x] Sierra Leone: No boosters based on https://africacdc.org/covid-19-vaccination/
• [x] Solomon Islands: SPC to release this data in the upcoming weeks
• [x] Somalia: No boosters based on https://africacdc.org/covid-19-vaccination/
• [x] South Sudan: No boosters based on https://africacdc.org/covid-19-vaccination/
• [x] Suriname: Added boosters in bc2bf0817cf76b920de93ca246d72d881ec4826d
• [x] Sudan: No boosters based on https://africacdc.org/covid-19-vaccination/
• [x] Syria: Boosters available 3046 also can be manually collected from https://www.facebook.com/MinistryOfHealthSYR/ (@Rouein)
• [x] Tanzania: No boosters based on https://africacdc.org/covid-19-vaccination/
• [x] Timor-Leste Boosters available but not extractable
• [x] Tonga: SPC to release this data in the upcoming weeks
• [x] Togo: No boosters based on https://africacdc.org/covid-19-vaccination/
• [x] Turkmenistan: No data available
• [x] Tuvalu: WHO to release this data in the upcoming weeks
• [x] Uganda: No boosters based on https://africacdc.org/covid-19-vaccination/
• [x] Uzbekistan: Boosters available but (ZF2001 being a 3 dose vaccine) booster numbers can not be extracted. WHO to release this data in the upcoming weeks.
• [x] Vanuatu: SPC to release this data in the upcoming weeks
• [x] Vietnam: Boosters added in #2487
• [x] Yemen: WHO to release this data in the upcoming weeks
• [x] Zambia Added in https://github.com/owid/covid-19-data/commit/0a6e98af98d0d37f40809c2c94dac1ee6d6e086b
• [x] Zimbabwe: Added in d1cbeb71404d63cca64be111f472dd9ea46fdabc

How to use this thread

Comment with the country you have examined and your conclusions. If applicable, state if you are taking the task to complete it. @lucasrodes will update this list to reflect your changes/updates.

Edit based on an original comment by @Rouein

Hi @edomt @camappel @Rouein

This work is based on the issue reported on https://github.com/owid/covid-19-data/issues/2301

The problem with the previous method is the new need for a hidden token inside the body, needed to process the date. Also I discovered the way how to get the valid dates and the values without scrapping, because all the data exist inside JavaScript variables and are easy to parse and convert to list values.

The new version support token extractor and valid dates for data use direct response to parse without scrapping. Also the new version calculate the "Positive Rate".

I migrate my code to your base class, I don't test over this version.

Sure, the extraction of the token and the values is possible to do with regular expression, but I did it quickly to be able to collaborate to attend to the high priority that you marked.

Important note, take the example of the interaction mechanism they had with the dates to make a similar mechanism. But in my tests I use multiprocessing with pool processing to speed up the times and the process takes much less time. I leave it to your discretion to evaluate incorporating parallelization. In the same way, migrating to regular expression if you think it is necessary.

Sorry if there are minor errors in the code, I promise in the future to install the OWID code to share something tested with your class model.

I hope to be helpful

Regards David

[High priority for the week of January 24]

• [x] https://github.com/owid/covid-19-data/pull/2307, https://github.com/owid/covid-19-data/pull/2317, https://github.com/owid/covid-19-data/pull/2367
• [x] https://github.com/owid/covid-19-data/pull/2318
• [x] https://github.com/owid/covid-19-data/pull/2302
• [x] Jamaica: not updated since 2022-01-08
• [x] https://github.com/owid/covid-19-data/pull/2305
• [x] https://github.com/owid/covid-19-data/pull/2313
• [x] https://github.com/owid/covid-19-data/pull/2308
• [x] https://github.com/owid/covid-19-data/issues/2261, https://github.com/owid/covid-19-data/issues/2319
• [x] https://github.com/owid/covid-19-data/pull/2310
• [x] #2320, #2322

JFYI:

Due to the webpage update, ROK publishes now the daily testing data instead of the cumulative one, and as OWID is probably only crawling this metric weekly, all the positivity rate and test date in OWID is not updated.

Official source: http://ncov.mohw.go.kr/en/bdBoardList.do?brdId=16&brdGubun=161&dataGubun=&ncvContSeq=&contSeq=&board_id=

Official test positivity metric is calculated as (covid case of the particular day) / (test numbers of the preceding day).

I do not know whether this goes against the principles of OWID, but these daily cumulative numbers are still recorded at Asia Regional Information Center at Seoul National University (TOTAL_TEST) on the first google sheet:

https://sites.google.com/view/snuaric/data-service/covid-19/covid-19-data

The data is manually collected through reviewing all the PDF files published weekly by the public health agency of Sweden. The data selected here are the most recent mentions of them in any of the published files.
Please note that the agency, until 2020-07-10 (Data pertaining to 29 June – 5 July) published only the number of people tested, therefore people tested label and from thence to 2020-10-02 ( Data of 21 – 27 September) published, only the number of the tests and thereafter both. Cumulative total is not mentioned in any of the publications, therefore the best estimate based on the weekly numbers is calculated. They have also published the data on antibody tests (conducted only in the first few weeks) and the antigen tests (currently conducted). Since these data were not used in their calculation of positive rates, they are not included here.

When displaying Israel's ICU data, current count stands on about 800, which corresponds to severely ill patients (in Israel defined as low blood oxygen, here translated from the dashboard by google-chrome as seriously ill). Actual number of patients in ICU is 244 (critical = system failure, lungs / heart / kidney). You can get the current number here, but unfortunately they stopped giving the history of this measure, and therefore no retro-corrections. https://datadashboardapi.health.gov.il/api/queries/hardPatient

People_fully_vaccinated has not been updated regularly since 10th Sept. for Indonesia. Last data updated on 18th Jan 2022. Data is present inWHO. Kindly look into the discrepancy.

In excess_mortality.csv data for El Salvador there's 1, 2 right after 9 in time column, and 2021-01-31 right after 2020-09-30 in date.

So there's missing 10, 11, 12 in time column and 2020-10-31, 2020-11-30, 2020-12-31 in date column.

This is a centralized issue to let our users suggest sources to add new countries, or new entries for existing countries, to our data on COVID-19 vaccinations.

• We only take into account numbers that are announced by official sources (head of government, ministry of health, public health agencies, public officials in charge of the vaccination campaign, etc.).
• We only count administered doses, not distributed doses.
• We will not include participants in the vaccine arm of clinical trials, as this data is not available for many of the hundreds of trials currently taking place.
• Our current sources are visible in vaccinations/locations.csv.

For countries that are already included in our data, most of our imports are automated and will collect the latest number when we update our dataset. If you don't see the latest number appear in our data, please wait at least 24 hours before suggesting it here.

Note that contributions via pull requests are not possible due to the way our data pipeline is set up.

Emoji system

@lucasrodes and @edomt use emojis to track comments in this issue.

• 👍 means that the comment has been read and looked into, and the data will be added during our next update (our vaccination dataset is updated each morning, London time)
• 👀 means that the comment has been read and looked into, but that the data (or some of the data) will not be added. This can be for a number of reasons: because we already have this data, because there is something wrong with the source, because the numbers contradict what other sources are showing, etc. If you have questions in this regard, you can read our contribute guideline or open a new issue referencing the comment.

Remarks

• Please read our "how to contribute" section for more details.
• This thread is meant to be used only to report new data points, for specific questions/suggestions please consider opening a new issue. If your comment is not a data report, a new thread may be opened to follow the discussion (see example).