For one, we lose the explicitly named New Zealand policies (Privacy Act, OIA, NZGOAL etc) however the ODC does mention to "observe domestic laws..." which eludes to the policy context here in NZ.

Perhaps if we did adopt the charter, we'd need some guide on the operationalisation of the ODC that would name these particular policies in use? Thoughts?

Another useful exercise I think would be to look at ODC (which is more comprehensive than NZDIMP) and identify aspects that would be generic core principles that all govt agencies should buy into, and recognise that other aspects may only be applicable at the higher government level (e.g. so individual organisations are not signing up to deliver upon things that they either cannot practically do, and/or that results in duplication of effort, or is simply not relevant to their situation...).

See also my comments at https://www.loomio.org/d/V84EyGBu/statistics-from-phase-1

@aaronmcglinchy cheers for the reply, so what you are saying is to look at having something more operational, where as the ODC is a more strategic level policy? :thumbsup:

Can you some suggestions of those aspects in the ODC you'd consider core and some examples of the aspects you think should be only applicable to higher government level as you suggest?

OK, responding to @camfindlay re my previous comments about some aspects of ODC not being applicable 'at the coal face'. And prefacing this by saying that I actually have no problems with ODC per se it reads as a a perfectly fine set of principles for a Government to adopt. My concerns come in contrasting ODC with NZDIMP, and the implication that ODC could/should outright replace NZDIMP. A key thing to understand my concern is that Government organisations are "directed" (Ministries/Departments) or "encouraged" (e.g. CRIs) to follow/adopt NZDIMP. The content of NZDIMP is such that there aren't any/many practical impediments to doing this. However ODC is more all encompassing, so if there was a similar directive/encouragement to follow ODC (in full), then that raises issues e.g.:

Principles 1-3 are largely OK/generic. There are some fine details that could be an issue in some contexts e.g. CRIs walking a tight-rope between making all date freely available, and govt requiring to operate like a company and make profit. No registration to access data - ODC seems to assume registration is a barrier, however for some of our collections we may require (free) registration so we can keep tabs on usage (and thus report on value of us making data freely available), can undertake the engagement with users that ODC encourages...

This part of 3 is not something an individual organisations can take responsibility for:

"Ensure wide data reuse through awareness raising and improved citizen data literacy."

Principle 4: Comparable and interoperable

• Data should be structured and standardised to allow comparison and interoperability across sectors, locations and time.
• Implement Open Standards data formats and common identifiers when collecting or publishing open data.
• Engage with domestic and global standards setting bodies to ensure new standards are interoperable.
• Map and publicly share local standards and identifiers to emerging global standards.

We do some work on standards, but much of this sort of stuff would need a National lead on it, rather than devolving to individual organisations (each of which will have greater or lesser understanding and capability...).

Lots of P5-6 are high level/out of scope for individual organisations providing the data.

Principle 5: For improved governance and citizen engagement

• Engagement and consultation with citizens helps government understand high demand data and improve data release prioritisation.
• Anticorruption information is released as open data.
• Provide training to increase capability of public servants to use open data for evidence based policy development.
• Align proactive release of open data with government obligations to release on request through engaging with freedom of information community.
• Engage with citizens and private sector to determine the data needed to effectively hold government accountable.
• Protect those using open data to identify corruption or criticise government.

Principle 6: For inclusive development and innovation

• Recognise the existence of the “digital divide” and limits on access and use of open data for marginalised people in society.
• Government actively support innovation based on reuse of open data.
• Encourage a richer open data ecosystem by empowering all sectors (citizens, businesses, academia, government employees) to release and innovate with valuable open data.
• Engage with education sector to improve data literacy curriculum.
• Share knowledge with other governments internationally and share technical expertise.
• Carry out research on the social and economic impacts of open data.
  

So, if the Govt adopts ODC, I think they would need something sitting beneath that to give guidance to individual organisations as to what the govt expects of them (and govt would need to acknowledge that it is taking on some of the specific responsibilities (cannot just say to everyone 'do it'...)).

Hi @aaronmcglinchy Your posts highlights to me one of the key differences between ODC and NZDIMP and why I believe the former should replace the latter - it's about the switch to a citizen-centric set of principles.

For example, with registration to receive data, I understand entirely your perspective that registration brings you benefits, but from a data user point of view it is a barrier for the following reasons:

• The first issue is one of principle, why should I have to give any identifier to use that data? I don't need to accept a license, I don't need to have my usage agreed, in fact I have the right to access that data entirely anonymously.
  
• The second is one of practicality, say I wish to automate access to the data and registration is required - does that mean I now need to get an API key to prove I've registered and what happens to the data around my access using that API key?

The ODC recognises that as a barrier because it takes a citizen-centric approach, which is so important for agencies to adopt regardless of what benefits they get from a more restrictive approach.

Also @aaronmcglinchy your recognition that Principle 4 of the ODC may require a national lead in some areas of data interoperability is, to my mind, a significant benefit of the ODC because

• Users of the data benefit by getting the data in a more joined up way so that they can look at it on the basis of the subject of the data regardless of the agency sources.

• Agencies that publish data benefit as it makes relating their data to that of other agencies much easier.

• A higher level of government benefits as it can now get a joined up picture of that related data across multiple agencies.

I agree, establishment of a national lead in some areas would be a good thing.

@camfindlay1 I've set up a test polis page here https://pol.is/5htvcrxtu9 which you might want to try out. It's a much easier way to derive consensus from a broader range of people and preserve minority opinions (see http://docs.pol.is/welcome/Overview.html). For example some people might get turned off by reading long comments here on loomio and the amount of time it takes to get up to speed (I know I was!), but polis's simplicity can give people with just 1 second or 1 minute of time to contribute.

At the moment I've only got the first 2 principles from the ODC seeded, and with anonymous voting enabled for full open access, but you might want to control that, I'd be happy to admin that poll, but I reckon you and your team should handle the moderation/authentication aspects of it. Let me know if you need any help setting things up.

Thanks @weijileong appreciate you taking the time to set that up. Let's not get too caught up in tooling at this stage. What's important here is the people aspect of participation, dialogue and knowing where to go to engage. Fragmenting the conversation further might mean people get less opportunities to interact with each other (because they are spread across different tools). In saying that, if you'd like to drive a citizen lead polis space and get some meaningful data, you could feed this back to us as input (even use it to inform your discussions here on Loomio?).

I was thinking of polis as more of a supplementary role than as a competitor, and yes, it's something that can be an interesting side project for now. The actual discussions would still take place at loomio of course, but I think that any tool that reduces friction in participating will have a place in implementing democratic policy. On another note, there is also the element of scaling up. How would you summarize all the discussions if there were hundreds if not thousands of participants? Would love to hear more of people's thoughts on this.

Agree about scaling @weijileong (that would be a good problem to have) and perhaps we can move the conversation of tools and process to somewhere like the Open Government Ninjas group. :thumbsup:

Back on the topic of this thread of looking at what is in the NZ Data and Information Management Principles that is NOT in the Open Data Charter, did you have any thoughts around this?

Just with the title of this thread, you might want to change it to something like: "Points in NZDIMP not found in ODC" to make it clearer for other people. Comparison as a word is a bit ambiguous.

I'll comment on ODC Principle 3 Accessible and usable cf. NZDIMP Readily available. The NZDIMP states "Open data and information are released proactively..." whereas the ODC principle only mentions data. If we define information as a higher derivative of data, this implies that NZDIMP covers more than just data, it has an extended coverage to information! For example, summary population statistics derived from the data (Statistics New Zealand being a good example) would have to be openly accessible as well, rather than just the raw data behind the scenes.

My opinion if I'm entitled one, is that this data and information part is something we should preserve. The data is important definitely, but we should also have open information so that people can more easily make sense of the underlying data without getting caught up with advanced statistical analysis that requires a university degree to work out.

Have you seen the comparison table @weijileong ? It points out those things that are the same, those things that are similar and those things in the ODC that are not in NZDIMP.

Thanks @weijileong and point taken on the title - I'll update. :star:

You also hit the nail on the head with regards to the impact on the information aspect. Stay tuned we'll be raising a new topic thread soon to pick through the impacts :smiley:

To your point about ability to work with data requiring a university degree. I note their are ODC principles around improving data literacy and awareness in both the public and among public servants alike :thumbsup:

Do you think if we all knew more about what we could do with open data and how to make more use of it in our everyday vocations that might be a positive impact the ODC enables (or at least puts this idea on the table)?

Yes, agreed that the ODC principles around improving engagement are sorely needed (see all the orange and red at Principles 5 and 6 of the ODC!). I'm not sure entirely what a picture of "everyday folks using open data to improve their lives" looks like, because I don't live in that world yet. But certainly though, if you have open information in addition to open data (making sure the link between the two/attribution is in place), I'm sure that can inspire people to interact with the data more.

P.S. you have a typo in your proposal, should be ODC not PDC :sweat_smile:

Thanks @weijileong looks like you can't change the proposal once people start voting (I can understand why too ;) ) - other than the typo feel free to lend your vote on this.

At the end of the day too, I think all these policy details should strive to be invisible, Just like tap water infrastructure, you don't want to know how the plumbing and all that works, you just want the water to be clean. Having that accessible information (water) is crucial, but sometimes people don't have to get into the details about the data (pipe infrastructure). Of course though, if something goes wrong (water leak), it's always good to have access to the plumbing :smiley:

We've started a new thread to discuss the impacts specifically drawing from the discussions over the last few weeks - please continue the discussions over at https://www.loomio.org/d/nRvY9XrW

Cam - From this page: https://www.ict.govt.nz/guidance-and-resources/open-government/new-zealand-data-and-information-management-principles/ I see no mention in the NZDIMP of specific NZ laws such as the Privacy Act. Am I looking at the right thing?

Well thought out response @aaronmcglinchy - exactly the kind of insight we are looking to draw out in this consultation :thumbsup:

@jaydaley I understand your perspective too, but to get to where you are talking about would take a shift at the Government level, not just organisational.

Taking the usefulness to us of registration, part of the reason for us is justifiying investment in the data and data delivery platforms. As a CRI our situation is different from say a Govt department, but in some respects I expect they'd have the same issue as I outline below.

We have a range of data that we manage, and limited resources with which to do so. Some data gets direct investment from govt, other data (or services or enhancements to the directly govt funded data) gets indirect/discretionary/and perhaps some commercial investment. Our management are challenged with making decisions about what areas to grow/support/invest in, and the curators of the data can help make the case for continued investment in particular data or delivery mechanisms... that benefits data users, but they need evidence to make that case, and basic information about registered users may help that case. So its a bit chicken and egg... and perhaps case specific, some data just inherently has recognised value and must be collected e.g. the Census data - if there is not a need to continually justify continued investment, then for such data perhaps registration is not necessary (from that perspective).

Also, ODC says things like users will know the limitations on the use of data... some data that we make available could be wrongly used and potentially have significant consequences. So there is some level of risk in us making data available (to both the user, and to us). Having some clear statements around the need to be aware of the appropriate use of data (e.g. understanding its degree of accuracy...) which are pointed out users when they first register helps us manage our risk, and is part of making users aware of limitations and ensuring they ask for advice if they may be using data for a purpose beyond its original applications/fitness for purpose... The alternative may be for data providers to be more risk averse and not provide data in as detailed a level or at all, which doesn't benefit users. Also, how do providers engage with users if they remain anonymous e.g. if errors were found in data how could we specifically advise people who have used the data?

I can't argue against the principle that people shouldn't have to identify themselves to use data, but there are practical reasons why it might benefit both parties. Basically I think that for both data users and data providers, it's a case of you can't (always) have your cake and eat it.

Thanks @aaronmcglinchy . I agree that this requires a shift but I'm not sure it's at the governmental level and not agency level. Only the other I was at a conference where Bill English said all the right things, such as "Anyone can know anything, now, for nothing" so it doesn't sound to me like a change is needed there.

At the agency level though it concerns me when you talk about management using evidence of use as part of the investment decisions regarding data. Open data is an obligation not a choice and evidence of use should not form any part of that.

Yes I agree that without registration agencies make take a more cautious approach to what detail is provided. Releasing data safely is not simple and a bit of caution at first is just an ordinary part of the learning cycle.

Some of the benefits that you see coming from the publication of data such as explaining issues with the data, are indeed important features in the way the data is released but they can achieved without registration. Data portals put a lot of effort into that problem.

Finally, just to note that i my experience the users of my published data come to me muc more frequently than I go to them

I was just re-reading some of this material and noticed your comment "At the agency level though it concerns me when you talk about management using evidence of use as part of the investment decisions regarding data. Open data is an obligation not a choice and evidence of use should not form any part of that."

Being obliged to do something, no mater how much you might agree with it, doesn't ensure that it happens (or even can happen). There are costs to collecting, maintaining, and sharing data. If management cannot see a demand or business case for particular data, then that data has got to be in the firing line whenever cost cutting measures are required.

There are differences between the likes of Departments/Ministries... and my experience in a CRI. The former may indeed be obligated to collect data on X and to share that data - they will also be funded to do that. For some data CRIs are specifically funded to collect and maintain data collections (though often funding is at a baseline level). Other data (or bells and whistles to existing data) is more discretionary, and unless there is an ongoing demand/funding source, the ongoing collection and/or availability of such data will always be subject to review. E.g. we recently reviewed the large number of websites we host that had arisen over the years, and some may now be retired. Repositories that maintain value by holding valued datasets can mean that less valued data housed within is also (still) available, but it will be the less well used data that is first to have further collection stopped...

Even for Depts/Ministries, if they are facing budget constraints, then I am sure that whilst they will strive to do what they are obliged to do, if funding is short, and some data is deemed to be less useful/less in demand, it would logically be that data which receives minimal resourcing to keep up to date/being collected and made available.

Basically I guess I am trying to say that the noble ideal of open data is well and good, but it is tempered against real world practicalities and resource constraints.

@jaydaley re "Only the other I was at a conference where Bill English said all the right things, such as “Anyone can know anything, now, for nothing” so it doesn’t sound to me like a change is needed there."

No doubt there will be /needs to be organisational change, but I think some tangible governmental change is required to facilitate that...

e.g. acknowledging govt needs to take the national level lead in some areas (investing in that aspect, and not just delegating it to multiple agencies and somehow it magically happens e.g. Australia have ANDS - we have no equivalent), change around expectations of profit from CRIs (if there is a genuine govt desire to make all data freely available, then there is a flipside... (and delivery costs))