Python Packaging at Kochi Python follow up

@sadiq can you try to fix http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=840174 ? npm2deb is written in python and its source is available at https://anonscm.debian.org/cgit/pkg-javascript/npm2deb.git

@suhail have contacted me regarding this bug. Hope he shall fix this. :)

There are many bugs, I asked him about another bug :) Can you generate dependency list for livescript? This is going like a chain reaction :(

I can't reproduce the bug you sent to @suhail [0].

I get the right result. I'm on Debian Stretch with no packges from sid. Let me know how to contact you via IRC. Or else, contact me at freenode (nick: pksadiq)

[0] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=840167

What is the output of apt-cache policy node-nomnom? node-nomnom was added to stretch only on 2016-08-03. Try again after you run apt-get update. I'm j4v4m4n on freenode.

I have contacted some folks and asked them to try fixing some bugs in npm2deb (its written in python). Please comment on the bugs if you have started working on any bugs.

Others please take one from http://bugs.debian.org/npm2deb and comment here before starting.

Already taken: #840167 (@suhail ) #840169 (Rahul Krishnan) # 840170 (@bobyaloysiusjohnso ) #840174 (@sadiq )

Available: #840213 #840207 #840226

Hi praveen ji,I am currently looking into the source code of npm2deb
https://github.com/LeoIannacone/npm2deb
to try and modify it to the pi to deb format as we had discussed that day.

If you can read Ruby code, please go through the source code of gem2deb also. From my experience, it automates more stuff than npm2deb.

It would be good to know the workflow gem2deb is following too.

@kvsreeram need a standard setup.py file i think :)

@sadiq Could you please update dependencies for browserify as well?

Done[0]. I shall try to upload the updated script soon. I have already updated the Template page.

[0] https://wiki.debian.org/Javascript/Nodejs/Tasks/Browserify

Please note that I'm running Debian testing (and not sid). So the dependency pasted in the wiki may not be up to date. Please inform me if you find any errors.

Thanks

If you need ssh access to a debian sid box, I can share our test machine.

Ranjith did not get his wiki account activated. So, I have updated the dependency creation script[0] and the template page[1]. Anyone interested may now help update the dependency wiki pages.

How to run:
1. download the script[0]
2. Install tree and npm: sudo apt install tree npm
3. run ./js_task_create.py module -o out-page
replace 'module' with the 'module' you wish. Eg: grunt
4. The file generated is saved to out-page (you can give any name for output file). Copy the complete file content. Edit the wiki page of the module, and replace the complete text.

Please note that the previous Template was dumb and listed the build dependencies under dependencies. I have fixed the template page. Please update the wiki pages.

Thanks

[0] https://wiki.debian.org/Javascript/Nodejs/Tasks?action=AttachFile&do=view&target=js_task_create.py
[1] https://wiki.debian.org/Javascript/Nodejs/Tasks/Template

Alignment needs manual fixing. Can you try to fix the script?

Is debian stretch or sid mandatory ?
Is jessie too old to get started ?

@shanavasm some tools (like npm2deb, dh-make, licensecheck) will be missing some features, but it is enough to get the concept. If you are willing to search and find workarounds for those issues go ahead. But I suggest you create an lxc container or virtual machine of sid inside jessie for packaging.

@suhail since we are not able to reproduce the bug you were working on, we can drop it for now. Can you try packaging string-width?

ok. I will try

@renjiththankachan can you try packaging lcid?

@praveenarimbrathod can you share a test debian box ?

Sure. We maintain ssh keys in https://gitlab.com/debian-ruby/pxq.in (private repo). Share your gitlab.com id and I'll give you access to this repo. Then you can add your ssh key there. If anyone else want a test box, follow the same steps.

Though a local sid environment can easily be created from any GNU/Linux distro by this command lxc-create -n debian -t debian -- -r sid. Once created, use lxc-attach -n debian. Create a normal user with adduser <username> and use su - to switch. See https://linuxcontainers.org/lxc/getting-started/ for more.

lxc done the job !

Have created a task page for lcid [1]. Debian package details of invert-kv (dependency of lcid) node-invert-kv (1.0.0-1) is missing from the generated output.
Is is need to be added manually or am I miss something ?

@shanavasm you need a task page only if the dependency list is big. Are you running the script from sid?

We can create the dependency tree locally with npm2deb like npm2deb depends -b -r lcid. If there is more than 10 unpackaged dependencies then only we need to create a task page.

@praveenarimbrathod yeah, I am running from sid. I shall try is-arrayish.

@shanavasm can you try is-arrayish?

@kvsreeram I think it would be better to package one lib before you try pypi2deb so you get a better idea of what you want to achive. Can you try packaging path-exists?

Will do from next week on praveenji

@praveenarimbrathod It seems there is a bug in js_task_create.py script. It gets the debian package details from the output of apt-cache madison <package> | grep Source (line no:46). But the Sources entry is included in the output only if there is a deb-src entry in sources.list. So the command should be apt-cache madison <package> | grep Packages. WDYT ?

Btw is-arrayish is a leaf package without any dependencies.

@shanavasm Haven't gone through code, but will rmadison solve the issue?

@balasankarchelamat rmadison lists details of package for all dists like stable, testing etc... . That makes parsing more complex.

@shanavasm So, you need info about a single distribution. You can use -s flag.

Am on mobile data, so can't look at the code now. What's it you are trying to achieve using the result? To check which version is available on sid?

@balasankarchelamat Yeah, it checks the corresponding debian package for node module.
apt-cache madison <package> | grep Packages fixes the issue. Just want to make sure it doesn't break anything else, so that the script in wiki can be updated.

Just consider the case of source and binary package having different names.

For example,
source package: node-mocha, binary package: mocha

@suhail @sadiq found root cause for #836182. I was missing deb-src line in my apt sources. Can you try changing it to look for binary packages or show a message asking user to enable deb-src instead of 'None'?

@praveenarimbrathod Checking for binary packages seems more reasonable. A hacky solution would be to use apt-cache madison <module> node-<module> | grep Packages. I have created a patch (attached) for npm2deb. I unable to test it because of #840811 [1].
Could you please verify if this patch works.
[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=840811

Have updated js_task_create.py too.

@praveen, I will try is-arrayish

@shanavasm use like 'rmadison -s sid node-jison'. You can test it with other packages which don't have utf-8 in description like jison. Submit the patch directly on https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=836182 and may be send a pull request to upstream github repo.

@praveenarimbrathod I have generated package and source files for is-arrayish as described in [1]. Files are at gitlab [2]. What next ?.
Shall I send an ITP ?

[1] https://wiki.debian.org/Javascript/Nodejs/Npm2Deb
[2] https://gitlab.com/shanavasm/debian-packaging/tree/master/is-arrayish

@shanavasm You should use gbp import-dsc command to import the package as a git repo. See https://wiki.debian.org/Diaspora/Packaging/gitlab#gbp_import-dsc

Unlike gem2deb, you'll have to download orig.tar.gz, run uupdate, and dpkg-source -b . to get the dsc file.

Go ahead with the ITP.

@sadiq I had to run these command in vim to clean the outfile.

1,$g/^node.*$/d
1,$g/^und.*$/d
1,$g/^colo.*$/d
1,$s/)$/) || || ||/

Can you check for packages in NEW and which has ITP bugs? npm2deb search does these already so you can refer its code.

@praveenarimbrathod I have created ITPs for is-arrayish [1] and strip-bom[2]. Repos are at https://gitlab.com/shanavasm/node-is-arrayish and https://gitlab.com/shanavasm/node-strip-bom

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=841158
[2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=841391

There is only one branch there. You should push all branches (git push --all) and all tags (git push --tags), but remove the debian/ only when it is ready for upload.

For node-strip-bom pristine-tar branch is missing. You should run gbp import-dsc --pristine-tar so we have pristine-tar data as well. pristine-tar branch is used to create exact orig.tar ball from upstream branch.

@praveenarimbrathod Thanks, Done :)

Once a package is ready for upload, you should change UNRELEASED to unstable (you can use dch -r command).

Also you should remove the line * New upstream release from debian/changelog. This was added by uupdate, since this is the initial release, we don't need that line. (uupdate is usually run only when we update the upstream release).

Also request membership to https://alioth.debian.org/projects/pkg-javascript group after signing up. We keep all our nodejs packages there so everyone in the team can update these packages.

It would also be a good idea to use a new dedicated email account for debian work as some of the mailing lists are high volume and also the email address becomes public. I suggest a community run email service like disroot.org, autistici.org, or riseup.net

@shanavasm Congratulations! Both packages are uploaded to debian! Now it will be reviewed by ftp masters before it is added to debian. They may ask you to change something, usually if you missed any copyright notices.

Also your request to join pkg-javascript on alioth is accepted. So next time, push your packages to alioth. See https://wiki.debian.org/Alioth/SSH for using ssh with alioth.

To create a new git repo you can follow these steps:

ssh git.debian.org
cd /git/pkg-javascript
./setup-repository node-is-arrayish 'Packaging for node-is-arrayish'


You can try is-equal-shallow next. You can take any leaf module from https://wiki.debian.org/Javascript/Nodejs/Tasks/grunt but just let me know before you start working so we don't duplicate work.

@praveenarimbrathod Thanks :):) !!.

I'll try is-equal-shallow next

Update https://wiki.debian.org/Javascript/Nodejs/Npm2Deb with more details so it becomes easy for others. Add the things you learned but missing from the wiki like requesting alioth membership, final update to debian/changelog, pushing all branches etc.

@praveenarimbrathod

npm2deb create is-equal-shallow throws the following error

Warnings occured:
[error] is-primitive: dependency node-is-primitive not in debian

How to handle dependencies not added to debian ?

It is recently packaged by @sruthi (https://tracker.debian.org/pkg/node-is-primitive). You get this error because your apt cache is not updated. If you use rmadison -s sid instead of apt-cache madison we can always get the latest info. Can you try to fix this in npm2deb https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=836182 ? Send a pull request to npm2deb github repo and send forwarded 836182 <pr link> to control at bugs.debian.org

My apt-cache was up to date. A little investigation shows that npm2deb searches only for 'source' packages and I don't have a source entry in my sources.list.

However rmadison seems to be perfect solution. I will try to fix it.

However rmadison seems to be perfect solution. I will try to fix it.

👍

I have fixed the package search issue [#836182]. Fix is available at https://github.com/shanavas786/npm2deb/tree/fix-package-search.

Thanks! Can you try fixing https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=840929 ?

#840929 was a local issue. Can you try https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=840226 ?

Fixed #840226.

https://github.com/LeoIannacone/npm2deb/pull/24

Looks good! Thanks :)

#840169 forwarded to https://github.com/LeoIannacone/npm2deb/pull/25

@praveenarimbrathod - same doubt as mentioned above

$ npm2deb create string-width
Warnings occured:
[error] is-fullwidth-code-point: dependency node-is-fullwidth-code-point not in debian
[error] strip-ansi: dependency node-strip-ansi not in debian

$ npm2deb depends -b -r string-width
Dependencies:
NPM

string-width (2.0.0)

├─ is-fullwidth-code-point (^2.0.0)

└─ strip-ansi (^3.0.0)

└─ ansi-regex (^2.0.0)

You have the same issue like @shanavasm , you have to update your apt cache with apt-get update.

$ npm2deb depends -b -r string-width
Dependencies:
NPM                                               Debian
string-width (2.0.0)                              None
├─ is-fullwidth-code-point (^2.0.0)               None
└─ strip-ansi (^3.0.0)                            node-strip-ansi (3.0.1-1)

Since is-fullwidth-code-point is not packaged, you have to do that first.

Seems is-fullwidth-code-point is very buggy https://lists.debian.org/debian-devel/2016/10/msg00743.html We have to use wcwidth.js instead. So please package that.

It has another dependency, defaults (^1.0.3), which is not packaged!

ok, start with defaults then.

dpkg-buildpackage is throwing following errors. node-clone (^1.0.2) is already packaged.

dpkg-gencontrol: warning: can't parse dependency node-clone (^1.0.2)
dpkg-gencontrol: error: error occurred while parsing Depends field:
, nodejs
, node-clone (^1.0.2)

You need to convert ^ to >=. See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=840207 after you complete this package, try fixing it in npm2deb so its converted automatically.

Trying to fix this[0]. Could you please check this[1]?

Testing on defaults package
CONTROL file before - [2]
CONTROL file now - [3]

[0] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=840207

[1] https://github.com/psuhailp/npm2deb/commit/4dc756ef5849bf51ee41938f33ca4680d6a0829b

[2] http://paste.debian.net/hidden/222a6fd2/

[3] http://paste.debian.net/hidden/dbe071e2/

Give a space after '>=', like '>= '.

You also need to look for n.x and ~. if version is '^1.x', it should be '>= 1.0'.

~1.0.0 means >= 1.0.0, << 1.1. ~1.0, means >= 1.0, << 2. You'll have to add two lines.

node-jju (~1.1.0) becomes
, node-jju (>= 1.1.0)
, node-jju (<< 1.2)

ok, i will change it.

isn't '^1.x' like '>= 1.0' & '<< 2'. I am confused :confused: . Is there any documentation on that?

https://docs.npmjs.com/misc/semver

Congratulations @suhail . Your package node-defaults is uploaded to debian.

Thanks :smiley:

node-defaults is accepted into unstable, so you can package wcwidth.js now.

Ok Praveen. I shall do wcwidth.js packaging next.

@sadiq @shanavasm @suhail @balasankarchelamat @bhe shall we do another packaging session in Kochi?

ya, sure