Open data standards

Hello Sam,

I think OpenFarm's current direction is fairly explicitly not Big Ag. As far as I understand us, we're aimed at small-scale farmers who don't have the means or resources of big agriculture companies (correct me if I'm wrong anyone). We're aiming to make food growing accessible and understandable to everyone, instead of just farmers, and pool common wisdom about plants.

Even if that's not yet the case, that's what I hope for, and would personally direct the project forwards in.

On defining a common API and data standard. We're pushing forward with building an incremental API based on OpenFarm's current needs. This is largely because we haven't found any "standard" model of defining things that is open and useable and meets our license requirements.

We came to this conclusion after numerous conversations about this kind of thing.

My gut reaction to the BigAg and OpenAg is that it smells of technological solutionism, one that classically tries to solve problems through technical solutions that might not be appropriate. The fact that this is coming from big agricultural companies only confirms that to me. Similarly, it seems to be a data standard about device communication, rather than actual plant information? I'd love to hear more opinions on this.

Thanks for linking to such a large group of folks that are dealing with the same stuff. I'm definitely interested in talking with you all.

Hi Simon, thanks for your thoughts, really helpful to see the links where you have already been discussing this stuff.

I realise that you are aimed at smaller growers (I am one myself and excited to see your project develop)

It seems like there are a lot of groups working in open source software for food, http://openfoodfoundation.org/document/open-source-food-projects-list-wiki/target-audience-functionality-web-based-openfood I realise they are diverse projects, with different aims, but as far as possible I'd like to see as little duplicated effort and as much compatibility between you as possible.

I just did this mind map which I'm not entirely sure is helpful: https://www.mindmup.com/#m:a1e7b5def0260401321ab002f3199dafb1 The green bits are supposed to be a life/distribution cycle of a particular carrot or potato.

I think your project sit's more in left hand side? With less interest in tracking any particular carrots life/ distribution cycle?

On the other hand box scheme software would sit more on the right, with a keen interest in the distribution side. I reckon it would be great if they could pull in data from your project, so consumers & growers could learn more about their food.

There is also the whole seed saving/ www.opensourceseedinitiative.org angle, which benefits from tracking actual germplasm/ plant/ seed cycles..

In terms of your project you seem closest to http://growstuff.org/ who are currently defining their API here: http://wiki.growstuff.org/index.php/API their data will be CC-BY-SA Is that license compatible with what you are hoping to do?

Thanks

Sam

Hi Sam,

Thanks for linking us to these other projects. I was not aware of them!

You're right that we're closest to what GrowStuff is building, though I think we have a specific niche even so: Crowdsourced and localized growing instructions. This means our API might look different from theirs because we just have different data. On the other hand, we do have some shareable data: crop/species information such as common names, common pests, etc.

However, licensing is a concern (unfortunately). At this point OpenFarm's data is going to be CC-BY, and it might become a mess if we intersperse our data with another source with another license such as GrowStuff's CC-BY-SA. Then there is license chaining to consider and everything becomes annoying and too red-tape-ey to be useful.

It would be great to ensure there is minimized double work, and awesome to see all of these projects sharing data and offering specific values to people. I'm not sure how to proceed and make that happen. Maybe an open invitation discussion/meeting to see where the viable opportunities for collaboration are and to pool efforts?

Hi Rory

Thanks for getting back. I have started to think about it as an ecosystem of FOSS software that can support various types of users: http://openfoodfoundation.org/document/open-source-food-projects-list-wiki/open-food-software-ecosystem

I think the work you are doing is vital, and I'm keen to see your data available to people running garden or farm information systems, or even box scheme software.

It feels like there are three main categories of data here:

1) Relatively fixed stuff: Latin & English Name, Variety, Taxonomic rankings, Minimum tolerated Temperature, Growing degree days, PH range, Rainfall range, Companion planting (in)compatibility, Common pests, diseases, Photo's.

2) Record keeping stuff, tracking the lives of specific plants and crops in specific geographical locations: Seed parentage information, variety, planting date, germination date, lat/long of plants, date plant care tasks undertaken, harvest date, yield, profit.

3) Information needed to get food distributed. There is a draft XML for this here being called #FoodChat http://openfoodfoundation.org/document/open-source-food-projects-list-wiki/boxomatic-xml-schema

I see these as somewhat hierarchical, as if you are running a food distribution system at (3) you might want to pull in the data about both how that Tomato has been grown (2), and also the taxonomic name etc. (1)

If I understand it correctly, your growing guides don't fit into this neatly, they seem to live between 1&2. The distillation of formal, or informal record keeping, into a coherent and geographically tagged bit of subjective reporting?

I think an on-line meeting would be a good idea, at some stage. There seems to be a lot of exciting activity in this area all of a sudden :)

I have been encouraging people to use #OpenFood as a shared Twitter tag for talking about this stuff.

I see what you are saying about licensing, maybe it's fine if you become the de-facto database for (1) type information.

If people want to grab your data and re-release it under CC-BY-SA is that permitted?

Cheers

Sam

I think you broke it down pretty well Sam, and I would add OpenFarm in as it's own category of data: subjective/creative ways of growing plants. So I have:

(1) Factual plant data that is always the same (this could be very easily shared between projects if there was a standard for it) (OpenFarm is pulling this data from http://www.ITIS.gov for now and could turn into an authority/de-facto down the road)
(1.5?) Data on where to get seeds, supplies, equipment sharing, etc
(2) Subjectve data on how to grow plants (OpenFarm)
(3) Record keeping of an individual plant in the real world (GrowStuff and other ag startups now, OpenFarm one day...)
(4) Data for moving individual plants to the consumer (#FoodChat, Open Food Foundation)
(4.5?) Subjective cooking and usage (OpenFarm maybe one day...)

^{^} This seems comprehensive and succinct as to what everyone in the space is doing and the type of data they are dealing with. It seems to provide clarity towards what open APIs might need to be developed and it provides ideas as to how the different categories/projects might interact. Would love to see if I missed anything or how someone else might break it down!

For licensing, one could take our data and re-release it under CC-BY-SA if they wanted, but it can't go back the other way. The SA license is less permissive and (in my opinion) less open because it places more limitation on the user. However, some say that ensuring openness with the SA is more open. It's a matter of perspective.

Hi Rory, yes this is starting to look good.

Just thinking about 1)

What about the case where new cultivar's emerge with more drought, or disease tolerance. (Perhaps as a result of Open Source Seed initiatives breeding). So the data is factual, and always the same, but also at any moment in time incomplete. So there needs to be a (peer review?) process of adding to that database?

I also recently came across this plant data .XML http://sourceforge.net/p/kitchengarden/gitcode/ci/master/tree/resources/species.xml Which I think is interesting in the way it maps relationships between species.

Just to also make a point on 3) one of my personal interests is open source seeds, and the emergence of new location adapted cultivars. This project will be much more interesting if records are kept (and linked) of the family history of these seeds. So if I live in a wet climate I may be really interested in the OS kale that has been bred in a wet climate for ten generations. I'm less interested in the kale that has been bred in a dry climate for ten generations. Both of these cultivars may have been bred from the same OS parent seed, but having the historical breeding/environment data is useful to me. This implies that the site where growing records are kept might also be a good place for the sale/exchange of OS seeds to take place..

I have more thoughts but I'm trying to make them (semi) coherent..

Thanks

Sam

I saw that XML file too - pretty cool work! It's almost as though there needs to be a social network of plants - who is friends with who, to what degree, and why?

I'm hoping that one day OpenFarm kind of taps into a lot of these data categories. For instance, in addition to the crops database, we could have a seeds database. We could build a garden management console like growveg.com where users virtually plant their garden, specifying which seeds they use and which guides they follow. The user gets an email once a week telling them what to do for each section of their garden based on the Guides, the seeds, and the other conditions.

Aggregate data could be really interesting as to where in the world each plant is being grown and how.

Hi folks,

I just thought I'd pop in and link this article which I shared with Rory a couple of weeks ago: http://wiki.growstuff.org/index.php/Importing_data This covers many of the issues related to importing data from one project to another, including the license chaining issues that Rory refers to upthread.

In my experience working on Freebase (http://freebase.com), for Google's knowledge group after they acquired Freebase, and on the early days of Wikidata (http://wikidata.org), I would suggest that the most important thing we can do is cross-reference each others' crops to make connections and interoperation between our datasets more effective. Or, if not connecting directly with each others' data sets, then at least with widely-used third party ones.

Growstuff already connects to Wikipedia which, by extension, connects us to Wikidata, Freebase, and then via Freebase to many other open data sources such as the Encyclopedia of Life.

I should probably write up something about the concepts behind this -- entities, unique identifiers, and semantic equivalence -- and post it somewhere. I'll drop a note here when I have done so.

@alexbayley Very interesting to connect to wikipedia to get some common identification!

I, for one, would be very interested if you do write up your experience with unique identifiers, semantic equivalence, etc.

OK, I just spent my morning writing something up, which I hope will be of interest: http://talk.growstuff.org/t/open-food-interoperability-entities-unique-ids-and-semantic-equivalence/93