mongolstats (development version)
-
nso_indicator_query()withoutperiodsnow selects every source period; the previous default sent an"all"selection that failed on fetch. -
nso_package()raisesmongolstats_offline_errorin offline mode instead of dropping every table with a warning. Withstrict = TRUEthe error now has classmongolstats_package_errorplus the package classes of the first failure (such asmongolstats_http_errorormongolstats_selection_error), rather than alwaysmongolstats_http_error. Results keep atbl_idcolumn even when empty. - Workers in
nso_package(parallel = TRUE)now overlap their downloads. The shared request lock is held only to reserve the next start slot, so request starts stay spaced while long transfers no longer block other workers or time out waiting for the lock. - Adding labels in
nso_data(),nso_dim_values()andnso_table_meta()can no longer duplicate rows when table metadata repeats a code or label. Mongolian labels that do not align with their codes are returned asNAinstead of raising an error. -
nso_data(),nso_search(),nso_dim_values(),nso_subsectors()and related functions rejectNAand blank identifiers before making requests. Validation errors from the indicator, portal, metadata and manifest functions name the function that was called. -
nso_offline_enable()andnso_offline_disable()return the previous setting invisibly, soold <- nso_offline_enable(); options(old)restores it. - The Mongolian site now mirrors all English articles, references, figures and navigation, with translation coverage checks and matching-page language links.
- Strengthened all analytical tutorials with checked observation keys, common calendar periods, strict missing-count aggregation and denominator validation.
- Qualified cancer, mortality, geographic and pollution interpretations; added explicit source/fallback coverage and conservative discrete funnel screening.
- Added reusable teaching helpers, offline analytical regression checks and a maintainer warning log for website builds.
mongolstats 0.4.0
CRAN release: 2026-10-31
- Experimental clients now expose the public data and metadata portals alongside PXWeb. The bundled data include an installed source, copyright, and license notice.
-
mn_fuzzy_join_by_name()limits distance calculations to one thread, including when stringdist has a larger process-wide default. -
nso_api_config()inspects server limits; NSO request attempts, retries, redirects, and worker processes share a scheduler with at least one second between attempts on one computer. The default retry delay also works when stats is not attached. -
nso_cache_enable()uses the standard R user cache directory, a 10 MiB size limit, and seven-day expiry by default. Setttl = NULLto disable age-based expiry. Existing caches in the former rappdirs location are retained and can be selected explicitly withdir. -
nso_check_data()reports duplicate observation keys, mixed units or geographic levels, unresolved source symbols, and absent requested periods without changing values. -
nso_data(),nso_fetch(), andnso_package()retain source provenance, resolved selections, request details, response metadata and comments, and raw observation symbols. Failed requests preserve available diagnostic context; unsupported multi-content responses raise explicit errors. -
nso_indicators(),nso_indicator(), andnso_indicator_query()provide reviewed GDP and resident-population query templates with definitions, units, caveats, and validation dates. -
nso_metadata(),nso_metadata_search(),nso_related(),nso_forms(),nso_classification(), andnso_metadata_dictionary()expose official definitions and collection context, preserving original dates, units, source identifiers, and quality flags. -
nso_portal_data()retrieves explicit public aggregate series, whilenso_portal_datamarts()discovers available datamarts. Source codes and official labels remain distinct; no automatic portal-to-PXWeb equivalence is assumed. -
nso_provenance(),nso_table_info(),nso_citation(),nso_write_manifest(), andnso_read_manifest()support inspecting and exporting source records.nso_value_status()preserves original observation symbols alongside numeric values. -
nso_search(live = TRUE)searches beyond the bundled catalogue.nso_catalogue_status()describes freshness and coverage; rebuilt indexes preserve build metadata and require explicit opt-in to replace the catalogue after an incomplete crawl. - Mapping documentation and validation use the released mongolmaps 0.2.0 or later from CRAN; the GitHub dependency override has been removed.
mongolstats 0.3.0
Breaking changes
- Time dimensions are now always returned by their labels (
"2024","2024-03"), whateverlabelsis. NSO codes periods by position, with"0"the latest period, so a code such asYear == "0"changed meaning every time NSO published a new period. Other dimensions are still returned as codes. Withlabels = "en"or"both"theYear_en-style columns are still added. -
nso_table_periods()now returns a tibble withcode,label,dateandfrequency, oldest period first, instead of a character vector of labels in catalogue order (newest first). It also finds time dimensions named “Month”, “Quarter” or in Mongolian, which it previously missed. -
nso_package(parallel = TRUE)now runs on mirai daemons throughpurrr::in_parallel()instead of future.apply; start workers withmirai::daemons(). future and future.apply are no longer suggested packages; mirai and carrier are. - sf has moved from Imports to Suggests: only the deprecated
mn_boundaries()needs it. - Discovery functions (
nso_dims(),nso_dim_values(),nso_itms_detail(),nso_sectors(),nso_subsectors(),nso_table_meta(),nso_table_periods()) now raisemongolstats_http_errorwhen a request fails. Previously any failure returned an empty result, so a server outage looked like an empty catalogue. Offline mode still returns empty results. - The
pxwebfallback innso_data(),nso_fetch(), andnso_package()has been removed, and pxweb is no longer a suggested package. The fallback returned data in a different shape (display-text column names, a value column that ignoredvalue_name), and it could make network requests in offline mode. Failed requests now raisemongolstats_http_errorwith the server’s response attached. -
mn_fuzzy_join_by_name()now errors whenmethod = "jw"is combined withmax_distance >= 1. Jaro-Winkler distances lie between 0 and 1, so the defaultmax_distance = 2matched every name to some boundary. Use a value such asmax_distance = 0.2. -
nso_options()now validates values (e.g.mongolstats.langmust be"en"or"mn",mongolstats.timeouta positive number) and leaves all options unchanged when one is invalid. It warns about names that are not mongolstats options. An unsupportedmongolstats.langset directly withoptions()now errors instead of silently falling back to English. -
nso_table_periods()now errors for an unknown table id or a non-stringtbl_id, like the other discovery functions. It previously returnedcharacter(0), which was indistinguishable from a table without a time dimension.
Lifecycle changes
- The geoBoundaries helpers
mn_boundaries(),mn_boundary_keys(),mn_boundaries_normalize(),mn_join_by_name()andmn_fuzzy_join_by_name()are deprecated in favour of the mongolmaps package, whose boundaries carry NSO codes and whosemn_join()joins NSO tables by code, drops totals, and takes Ulaanbaatar’s figures from region code"5"when a table leaves"511"empty. -
nso_itms(),nso_itms_detail()andnso_itms_search()are superseded bynso_tables(),nso_variables()andnso_search(fixed = TRUE). They keep working without warnings.
New features
- Tables that NSO has moved to another catalogue folder are now found automatically: when a table’s metadata request fails with HTTP 400 or 404, mongolstats looks it up in the live catalogue, updates the in-memory index, and tells you (class
mongolstats_table_moved). Tables published after the bundled index was built are found with the PXWeb search API (classmongolstats_table_located). Previously both failed, e.g. the GDP tableDT_NSO_0500_001V1after NSO’s September 2026 reorganisation. - Requests larger than the server’s limit of 1,000,000 cells are split into several requests and the results combined. The limit is set by the new option
mongolstats.max_cells. - All requests are rate-limited to data.1212.mn’s published limit of 1,000 calls per 100 seconds.
- New
nso_latest_periods()returns the labels of a table’s most recent periods, ready forselections. - New
nso_period_date()converts NSO period labels in any of the office’s formats ("2024","2024-03","2016M3","2024.03","2024-II","2026-07-06") to dates. - Selections on the time dimension match periods written in any of these formats, so
nso_period_seq("202401", "202412", by = "M")now selects months labelled"2024-01"or"2024M1". Previously its output matched no NSO monthly table. - Labels in
selectionsmatch ignoring the leading spaces NSO uses to indent nested categories, e.g."GDP, at 2015 constant prices"for" GDP, at 2015 constant prices". -
nso_search()gainsfixed = TRUEfor literal matching. -
nso_tables()gains anupdatedcolumn with the date NSO last updated each table, and the bundled table index has been rebuilt from the current catalogue. - Table ids are now matched case-insensitively, with or without the
.pxsuffix (e.g."dt_nso_0300_001v2"). -
mn_join_by_name()andmn_fuzzy_join_by_name()now warn (classmongolstats_unmatched_names) naming data rows that matched no boundary. Previously these rows were dropped silently, so a spelling mismatch showed up only as a grey polygon on a map. -
nso_rebuild_px_index()warns (classmongolstats_incomplete_index) naming catalogue paths whose requests failed, instead of silently writing an incomplete index, and stops immediately in offline mode.
Bug fixes
- The start and end periods in the table index (
strt_prd,end_prd) were reversed for most tables, because NSO lists periods newest first, and missing for tables whose time dimension is not named “Year” or “Time”. They are now the earliest and latest periods by date. -
nso_itms_by_sector()andnso_search(sector = )now include tables in sub-folders. NSO keeps every table in a sub-folder, so a sector id fromnso_sectors()previously matched no tables. -
nso_dims()andnso_table_meta()now mark the time dimension inis_time. NSO never sets PXWeb’s time flag, sois_timewas alwaysFALSE. - HTTP errors no longer fail with “Could not evaluate cli
{}expression” when a server or curl message contains a brace. The original error is now attached as the parent condition. - Argument validation errors now describe the offending value (“not
NULL”, “not a number”). They used a nonexistent cli style and printed the raw value instead, e.g. “not .” forNULL. -
mn_boundaries()raisesmongolstats_http_errorwhen the boundary download fails, and a clear error when the GeoBoundaries API returns no download URL. -
mn_fuzzy_join_by_name()no longer crashes with “invalid subscript type ‘list’” when the name column containsNA. -
nso_cache_enable(): cached catalogue listings and metadata are now keyed by the PXWeb base URL and database, so switchingmongolstats.px_base_urlormongolstats.px_dbno longer returns another server’s metadata. The table index is no longer stored in the disk cache, where a stale copy outlived package upgrades that refresh the embedded index. -
nso_data()andnso_fetch()in offline mode now raisemongolstats_offline_error. When table metadata was cached they raisedmongolstats_http_error, so handlers could not tell offline mode from a network failure. -
nso_data(),nso_fetch(), andnso_package()withlabelsother than"none"now error when the label metadata request fails. They previously returned the data without labels and without a warning. Innso_package()the table is reported as failed. -
nso_itms_detail()andnso_variables()now return Mongolian labels inscr_mn. Labels were joined on the language-specific dimension name, which never matches across languages, soscr_mnwas alwaysNA. -
nso_itms_by_sector()validateslist_idinstead of failing inside a subsetting call. -
nso_package(parallel = TRUE)now applies the caller’s mongolstats options (language, offline mode, timeouts, base URL) inside each worker. Workers previously ran with the package defaults. -
nso_package()validatesrequestsbefore fetching and explains the problem, e.g. a single record passed without wrapping it inlist(), a record withouttbl_id, or non-listselections. These previously failed with errors such as “$ operator is invalid for atomic vectors”. -
nso_search()no longer changes the meaning of regex escapes: the pattern was lowercased for case-insensitive matching, turning\Sinto\s.nso_search()andnso_itms_search()now match withignore_case = TRUEinstead.
Performance
-
nso_data()flattens large PXWeb responses much faster: row keys are collected into one matrix instead of binding a data frame per row (about 8x faster on a 50,000-row response). Malformed responses whose row keys do not match the dimension columns now raise an error.
Internal
- Recorded HTTP fixtures now use short paths. The old paths exceeded the 100 bytes
R CMD buildstores portably, so the fixtures were dropped from the built package and the recorded tests always skipped. Those tests replay without network access and no longer skip on CRAN. - All top-level hidden files and folders (such as a local
.venv/) are excluded from package builds. - The pkgdown site no longer publishes a Markdown copy of every page.
- Requests share one builder (user agent, timeout, retries, rate limit), and HTTP errors carry the response
status. - The vignettes and README use
mongolmaps::mn_join()without recoding Ulaanbaatar by hand, select periods by label, and follow NSO’s renamed tables and dimensions. The monthly mortality article moved tovignettes/articles/. - Continuous integration now fails on lints, on test coverage below 80% (tests that need the live API skip in CI), and on out-of-date Rd files, instead of regenerating them before the check. The package, tests, and vignettes are lint-clean.
- Error-message tests use snapshots, and new offline tests cover the discovery functions against a fake table.
mongolstats 0.2.0
CRAN release: 2026-07-09
Breaking changes
-
Stricter selection validation:
nso_data(),nso_package(),nso_query(), andas_px_query()now error when a selection name does not match any dimension in the table (previously the selection was silently ignored and the full dimension was fetched), when selections are unnamed, or when two selections target the same dimension. This prevents silently fetching wrong-scope data. Errors carry classmongolstats_selection_error.
New features
-
nso_package()no longer fails silently: tables that fail to fetch are reported in a warning naming each table, and a newstrict = TRUEargument raises an error instead. Previously failed tables were dropped without any signal. -
Unified
labelsvocabulary:nso_data()andnso_package()accept"code"as an alias for"none", andnso_fetch()andnso_dim_values()accept"none"as an alias for"code", so either spelling works everywhere. -
mn_boundaries()session cache: boundary downloads are cached in memory for the session (repeated calls, including viamn_join_by_name(), no longer re-download multi-megabyte GeoJSON). Arefresh = TRUEargument bypasses the cache. -
nso_period_seq()input validation: malformed periods (wrong width, month 13,NA, vectors) and reversed ranges now error with a clear message instead of silently returning a descending sequence or failing insideseq.Date().
Bug fixes
Ambiguous table ids now warn: the NSO catalogue can reuse one table id for different tables in different folders (e.g.
DT_NSO_2400_015V1currently names both the monthly SO2 station table and an annual air-quality-standards table). mongolstats now warns (classmongolstats_ambiguous_table) and names the folder it picked instead of silently resolving to an arbitrary table. Tables that are merely cross-listed in several folders under the same name resolve silently.The embedded PXWeb table index has been refreshed to match the current NSO catalogue. Several tables (e.g. cancer incidence
DT_NSO_2100_012V1and communicable diseasesDT_NSO_2100_035V1) moved to different catalogue folders, which made lookups through the stale index return empty results.labels = "mn"andlabels = "both"now actually attach Mongolian labels: label metadata was matched to data columns by display text, but the two languages only share the dimension code (the English column isSexwhile both languages use the codeХүйс), so cross-language labels were silently never added. Labels are now routed through a code-to-column map, which also fixes English labels whenmongolstats.lang = "mn".Offline mode no longer fires the session-cookie GET:
.px_session_cookie()bypassed the offline gate, sonso_data()with cached metadata still performed one live request in offline mode.Retries no longer crash on the first backoff: the default retry backoff formula referenced
..attempt, which does not exist in rlang lambdas (the argument is.x), so any transient response (429/503) failed with “object ‘..attempt’ not found” instead of retrying.HTTP error and verbose messages now include the request URL: an internal misuse of
httr2::req_url()as a getter meant the URL always rendered asNAand was dropped from messages.Time columns typed
"t"are handled: the PXWeb response flattener kept only"d"-typed columns; a server typing its time column"t"(the PXWeb convention; NSO currently uses"d") would have misaligned row keys with column names.nso_fetch()now honors themongolstats.default_labelsoption, matchingnso_data().nso_itms_search()matches its query literally as documented (“keyword”); regex metacharacters (e.g."c++") no longer error.nso_search()remains a regex search.nso_period_seq()rejects trailing garbage after a valid year (e.g."2024abc","2024-06"); previously the year prefix was silently used.mn_join_by_name()andmn_fuzzy_join_by_name()error clearly whenname_colis not a column ofdata, instead of failing with an obscure replacement-length error.nso_table_periods()finds the time dimension forlang = "mn"by also matching the Mongolian display name.Mixed codes and labels: Selection values are now mapped element-wise, so codes and labels can be mixed in one vector (e.g.
Sex = c("Total", "1")). Previously this produced an “Unknown value: NA” error. Duplicated labels in table metadata now produce a warning naming the ambiguous label.Consistent errors:
as_px_query()now raises the same classed errors asnso_data()for invalid selections (previously a plainstop()).Correct error classes: network failures (DNS, timeout) now signal
mongolstats_http_error; previously they were misclassified asmongolstats_offline_error, so handlers could not tell offline mode from a genuine connection problem.
Performance
- The PXWeb session cookie is now seeded once per base URL per session instead of once per data fetch, removing one GET round trip from every
nso_data()call after the first. -
mn_fuzzy_join_by_name()no longer computes a base-R distance matrix that was immediately discarded when stringdist is installed.
CRAN compliance
- The cache test no longer writes to (or clears) the user’s real cache directory: it now skips when the cache packages are missing and uses a throwaway directory under
tempdir(). Previously it ran on CRAN and touchedrappdirs::user_cache_dir("mongolstats"). - The
nso_cache_enable()example is now conditional on memoise/cachem/rappdirs being installed. - Network-dependent examples are now guarded with
NOT_CRANin addition tocurl::has_internet(), so they no longer contact the live PXWeb API during CRAN checks. Thenso_rebuild_px_index()example, which crawls the full catalogue, is wrapped in\dontrun{}. -
inst/extdata/air_monthly_cached.csvis now gzip-compressed (6.4 Mb to 0.3 Mb), bringing the installed package size well under CRAN’s threshold.read.csv()decompresses it transparently; only the vignette referenced it.
Internal
- Removed the unused legacy HTTP layer (
.nso_req(),.nso_get(),.nso_post()and friends) together with the now-inertmongolstats.base_urloption; all requests go to the PXWeb API base (mongolstats.px_base_url). -
nso_search()andnso_itms_search()share one search helper; the PXWeb response flattener was factored into.px_flatten_response()and unit-tested; themongolstats.paralleloption is now registered and reported bynso_options(). - The GeoBoundaries metadata request now uses the same timeout/retry policy as other requests.
- Selection mapping and validation consolidated into a single helper (
.px_map_selections()), removing three divergent copies of the logic. - Table resolution now goes through
.px_resolve_table()everywhere (.px_add_labels(),nso_table_periods(),.px_build_body()), removing the remaining inlined index lookups. - Integration tests resolve dimension codes from live metadata instead of assuming positional codes, so tests no longer break when NSO adds a new period. Integration and endpoint tests now assert concrete shapes and no longer swallow errors.
- Added
tools/record_fixtures.Rto record the httptest2 fixtures used bytest-recorded-*.R; those tests previously always skipped because the fixture directories were empty. - Removed unused
globalVariables()declarations, the unused lifecycle Suggests entry, and a stray rendered vignette HTML from the repository.
mongolstats 0.1.1
CRAN release: 2026-01-26
Patch release to fix CRAN check failures on vignette rebuilding.
mongolstats 0.1.0
CRAN release: 2026-01-18
First minor release containing significant data updates, visualization improvements, and refined documentation.
Data & Features
-
Monthly Mortality Data: Switched infant mortality analysis to use monthly data (
DT_NSO_2100_015V1) for 2019-2024, enabling more granular trend analysis and up-to-date regional comparisons. - UB Air Quality: Added accurate coordinates for Ulaanbaatar air quality monitoring stations and mapped them to districts.
- Boundaries: Fixed spatial mismatches in administrative boundary names to ensure seamless joins with NSO data.
Visualization & UI
-
Plot Polish: Applied publication-quality styling to all package plots, including:
- Interactive tooltips with formatted units and rounded numbers.
- Trend lines with confidence intervals for mortality rates.
- Log-scaled population maps for better contrast.
- Clean integer breaks on axes.
- Standardized Aesthetics: Consistent plot styling across all vignettes.
Documentation
-
New Vignettes:
-
mortality-analysis: In-depth analysis of infant mortality trends and seasonality. -
environmental-surveillance: Detailed examination of air and water quality monitoring coverage.
-
-
Vignette Updates:
-
getting-started: Renamed and streamlined for new users. -
discovery&mapping: Updated with new datasets and visualization techniques.
-
-
Examples: Added executable examples to key functions (
nso_data(),nso_itms(),mn_boundaries()).
mongolstats 0.0.0.9000
Initial development snapshot.
- NSO API wrappers:
nso_itms(),nso_itms_detail(),nso_data(),nso_package() - Sector helpers:
nso_sectors(),nso_subsectors(),nso_search() - Boundaries:
mn_boundaries(), join helpers incl. fuzzy joins - Caching for discovery endpoints
- Vignettes for getting started, mapping, fuzzy joins, and code-based joins