Thanks to visit codestin.com
Credit goes to github.com

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 23 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ Leave it null for the first run of the script. Then the script will show you whi

#### gitlab.listArchivedProjects

When listing projects on the first run (projectID = null), include archived ones too. The default is *true*.
When listing projects on the first run (projectID = null), include archived ones too. The default is `true`.

#### gitlab.sessionCookie

Expand All @@ -108,25 +108,25 @@ Under which organisation or user will the new project be hosted

#### github.ownerIsOrg

A boolean indicator (default is *false*) to specify that the owner of this repo is an Organisation.
A boolean indicator (default is `false`) to specify that the owner of this repo is an Organisation.

#### github.token

Go to [Settings / Developer settings / Personal access tokens](https://github.com/settings/tokens). Generate a new token with `repo` scope and copy that into the `settings.ts`

#### github.token_owner

Set to the user name of the user whose token is used (see above). This is required to determine whether the user running the migration is also the creator of comments and issues. If this is the case and `useIssueCreationAPI` is true (see below), the extra line specifying who created a comment or issue will not be added.
Set to the user name of the user whose token is used (see above). This is required to determine whether the user running the migration is also the creator of comments and issues. If this is the case and `useIssueCreationAPI` is `true` (see below), the extra line specifying who created a comment or issue will not be added.

#### github.repo

What is the name of the new repo

#### github.recreateRepo

If true (default is false), we will try to delete the destination github repository if present, and (re)create it. The github token must be granted `delete_repo` scope. The newly created repository will be made private by default.
If `true` (default is `false`), we will try to delete the destination github repository if present, and (re)create it. The github token must be granted `delete_repo` scope. The newly created repository will be made private by default.

If you've set `github.recreateRepo` to true and the repo belongs to an Organisation, the `github.ownerIsOrg` flag **must** be set as true.
If you've set `github.recreateRepo` to `true` and the repo belongs to an Organisation, the `github.ownerIsOrg` flag **must** be set as `true`.

This is useful when debugging this tool or a specific migration. You will always be prompted for confirmation.

Expand Down Expand Up @@ -160,63 +160,67 @@ When one renames the project while transfering so that the projects don't loose

#### conversion.useLowerCaseLabels

If this is set to true (default) then labels from GitLab will be converted to lowercase in GitHub.
If this is set to `true` (default) then labels from GitLab will be converted to lowercase in GitHub.

#### conversion.addIssueInformation

If this is set to `true` (default) then issues and pull requests will get information about assignees (both), reviewers and approvers (PR only) added to their description.

### transfer

#### transfer.milestones

If this is set to true (default) then the migration process will transfer milestones.
If this is set to `true` (default) then the migration process will transfer milestones.

#### transfer.labels

If this is set to true (default) then the migration process will transfer labels.
If this is set to `true` (default) then the migration process will transfer labels.

#### transfer.issues

If this is set to true (default) then the migration process will transfer issues.
If this is set to `true` (default) then the migration process will transfer issues.

#### transfer.mergeRequests

If this is set to true (default) then the migration process will transfer merge requests.
If this is set to `true` (default) then the migration process will transfer merge requests.

#### transfer.releases

If this is set to true (default) then the migration process will transfer releases.
If this is set to `true` (default) then the migration process will transfer releases.
Note that github api for releases is limited and hence this will only transfer the title and description of the releases
and add them to github in chronological order, but it would not preserve the original release dates, nor transfer artefacts or assets.

### dryRun

As default it is set to false. Doesn't fire the requests to github api and only does the work on the gitlab side to test for wonky cases before using up api-calls
As default it is set to `false`. Doesn't fire the requests to github api and only does the work on the gitlab side to test for wonky cases before using up api-calls

### exportUsers

If this is set to true (default is false) then a file called "users.txt" wil be created containing all
If this is set to `true` (default is `false`) then a file called "users.txt" wil be created containing all
usernames that contributed to the repository. You can use this with dryRun when you need to map users
for the migration, but you do not know all the source usernames.

### useIssueImportAPI

Set to true (default) to enable using the [GitHub preview API for importing issues](https://gist.github.com/jonmagic/5282384165e0f86ef105). This allows setting the date for issues and comments instead of inserting an additional line in the body.
Set to `true` (default) to enable using the [GitHub preview API for importing issues](https://gist.github.com/jonmagic/5282384165e0f86ef105). This allows setting the date for issues and comments instead of inserting an additional line in the body.

### usePlaceholderIssuesForMissingIssues

If this is set to true (default) then the migration process will automatically create empty dummy issues for every 'missing' GitLab issue (if you deleted a GitLab issue for example). Those issues will be closed on Github and they ensure that the issue ids stay the same on both GitLab and Github.
If this is set to `true` (default) then the migration process will automatically create empty dummy issues for every 'missing' GitLab issue (if you deleted a GitLab issue for example). Those issues will be closed on Github and they ensure that the issue ids stay the same on both GitLab and Github.

#### usePlaceholderMilestonesForMissingMilestones

If this is set to true (default) then the migration process will automatically create empty dummy milestones for every 'missing' GitLab milestone (if you deleted a GitLab milestone for example). Those milestones will be closed on Github and they ensure that the milestone ids stay the same on both GitLab and Github.
If this is set to `true` (default) then the migration process will automatically create empty dummy milestones for every 'missing' GitLab milestone (if you deleted a GitLab milestone for example). Those milestones will be closed on Github and they ensure that the milestone ids stay the same on both GitLab and Github.

#### useReplacementIssuesForCreationFails

If this is set to true (default) then the migration process will automatically create so called "replacement-issues" for every issue where the migration fails. This replacement issue will be exactly the same, but the original description will be lost. In the future, the description of the replacement issue will also contain a link to the original issue on GitLab. This way, users who still have access to the GitLab repository can still view its content. However, this is still an open task. (TODO)
If this is set to `true` (default) then the migration process will automatically create so called "replacement-issues" for every issue where the migration fails. This replacement issue will be exactly the same, but the original description will be lost. In the future, the description of the replacement issue will also contain a link to the original issue on GitLab. This way, users who still have access to the GitLab repository can still view its content. However, this is still an open task. (TODO)

It would of course be better to find the cause for migration fails, so that no replacement issues would be needed. Finding the cause together with a retry-mechanism would be optimal, and will maybe come in the future - currently the replacement-issue-mechanism helps to keep things in order.

### useIssuesForAllMergeRequests

If this is set to true (default is false) then all merge requests will be migrated as GitHub issues (rather than pull requests). This can be
If this is set to `true` (default is `false`) then all merge requests will be migrated as GitHub issues (rather than pull requests). This can be
used to sidestep the problem where pull requests are rejected by GitHub if the feature branch no longer exists or has been merged.

### filterByLabel
Expand All @@ -238,7 +242,7 @@ Suggested values:

### mergeRequests

Object consisting of `logfile` and `log`. If `log` is set to true, then the merge requests are logged in the specified file and not migrated. Conversely, if `log` is set to false, then the merge requests are migrated to GitHub and not logged. If the source or target branches linked to the merge request have been deleted, the merge request cannot be migrated to a pull request; instead, an issue with a custom "gitlab merge request" tag is created with the full comment history of the merge request.
Object consisting of `logfile` and `log`. If `log` is set to `true`, then the merge requests are logged in the specified file and not migrated. Conversely, if `log` is set to `false`, then the merge requests are migrated to GitHub and not logged. If the source or target branches linked to the merge request have been deleted, the merge request cannot be migrated to a pull request; instead, an issue with a custom "gitlab merge request" tag is created with the full comment history of the merge request.

### usermap

Expand Down
3 changes: 2 additions & 1 deletion sample_settings.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ export default {
accessKeyId: '{{accessKeyId}}',
secretAccessKey: '{{secretAccessKey}}',
bucket: 'my-gitlab-bucket',
region: null,
region: 'us-west-1',
},
usermap: {
'username.gitlab.1': 'username.github.1',
Expand All @@ -34,6 +34,7 @@ export default {
},
conversion: {
useLowerCaseLabels: true,
addIssueInformation: true,
},
transfer: {
description: true,
Expand Down
45 changes: 41 additions & 4 deletions src/githubHelper.ts
Original file line number Diff line number Diff line change
Expand Up @@ -346,7 +346,9 @@ export class GithubHelper {
let bodyConverted = await this.convertIssuesAndComments(
issue.description ?? '',
issue,
!this.userIsCreator(issue.author) || !issue.description
!this.userIsCreator(issue.author) || !issue.description,
true,
true,
);

let props: RestEndpointMethodTypes['issues']['create']['parameters'] = {
Expand Down Expand Up @@ -460,7 +462,9 @@ export class GithubHelper {
: await this.convertIssuesAndComments(
issue.description ?? '',
issue,
!this.userIsCreator(issue.author) || !issue.description
!this.userIsCreator(issue.author) || !issue.description,
true,
true,
);

let props: IssueImport = {
Expand Down Expand Up @@ -985,7 +989,10 @@ export class GithubHelper {
if (canCreate) {
let bodyConverted = await this.convertIssuesAndComments(
mergeRequest.description,
mergeRequest
mergeRequest,
true,
true,
true,
);

// GitHub API Documentation to create a pull request: https://developer.github.com/v3/pulls/#create-a-pull-request
Expand Down Expand Up @@ -1027,7 +1034,9 @@ export class GithubHelper {
let bodyConverted = await this.convertIssuesAndComments(
mergeStr + mergeRequest.description,
mergeRequest,
!this.userIsCreator(mergeRequest.author) || !settings.useIssueImportAPI
!this.userIsCreator(mergeRequest.author) || !settings.useIssueImportAPI,
true,
true,
);

if (settings.useIssueImportAPI) {
Expand Down Expand Up @@ -1284,12 +1293,15 @@ export class GithubHelper {
* @param str Body of the GitLab note
* @param item GitLab item to which the note belongs
* @param add_line Set to true to add the line with author and creation date
* @param add_line_ref Set to true to add the line ref to the comment
* @param add_issue_information Set to true to add assignees, reviewers, and approvers
*/
async convertIssuesAndComments(
str: string,
item: GitLabIssue | GitLabMergeRequest | GitLabNote | MilestoneImport | GitLabDiscussionNote,
add_line: boolean = true,
add_line_ref: boolean = true,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't used in this PR, no?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My bad, I accidentally copied this change over from my own branch that contains all improvements in one. It belongs to #210. Now that that one is merged and this one up to date it is fixed.

add_issue_information: boolean = false,
): Promise<string> {
// A note on implementation:
// We don't convert project names once at the beginning because otherwise
Expand Down Expand Up @@ -1464,6 +1476,11 @@ export class GithubHelper {
this.gitlabHelper
);

if (add_issue_information && settings.conversion.addIssueInformation) {
let issue = item as GitLabIssue;
str = await this.addIssueInformation(issue, str)
}

if ('web_url' in item) {
str += '\n\n*Migrated from GitLab: ' + item.web_url + '*';
}
Expand Down Expand Up @@ -1564,6 +1581,26 @@ export class GithubHelper {
return lineRef;
}

async addIssueInformation(issue: GitLabIssue | GitLabMergeRequest, description: string): Promise<string> {
let bodyConverted = description;

let assignees = issue.assignees.map(a => a.username) as string[];
bodyConverted += utils.organizationUsersString(assignees, "Assignees");

// check whether issue is of type GitLabMergeRequest
if (issue.reviewers) {
let mergeRequest = issue as GitLabMergeRequest;

let mrReviewers = mergeRequest.reviewers.map(a => a.username) as string[];
bodyConverted += utils.organizationUsersString(mrReviewers, 'Reviewers');

let approvals = await this.gitlabHelper.getMergeRequestApprovals(mergeRequest.iid);
bodyConverted += utils.organizationUsersString(approvals, 'Approved by');
}

return bodyConverted;
}

/**
* Meh...
* @param milestoneMap
Expand Down
22 changes: 22 additions & 0 deletions src/gitlabHelper.ts
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,28 @@ export class GitlabHelper {
return this.allBranches as any[];
}

async getMergeRequestApprovals(pullRequestIid: number): Promise<string[]> {
try {
let approvals = await this.gitlabApi.MergeRequestApprovals.showConfiguration(
this.gitlabProjectId,
{
mergerequestIId: pullRequestIid,
},
);

if (approvals.rules[0]) {
return approvals.rules[0].approved_by.map(user => user.username);
}

console.log(`No approvals found for GitLab merge request !${pullRequestIid}.`)
} catch (err) {
console.error(
`Could not fetch approvals for GitLab merge request !${pullRequestIid}: ${err}`
);
}
return [];
}

/**
* Gets all notes for a given merge request.
*/
Expand Down
1 change: 1 addition & 0 deletions src/settings.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ export default interface Settings {
};
conversion: {
useLowerCaseLabels: boolean;
addIssueInformation: boolean;
};
transfer: {
description: boolean;
Expand Down
20 changes: 20 additions & 0 deletions src/utils.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { S3Settings } from './settings';
import settings from '../settings';
import * as mime from 'mime-types';
import * as path from 'path';
import * as crypto from 'crypto';
Expand Down Expand Up @@ -97,3 +98,22 @@ export const migrateAttachments = async (
({}, {}, {}, {}, offset, {}) => offsetToAttachment[offset]
);
};

export const organizationUsersString = (users: string[], prefix: string): string => {
let organizationUsers = [];
for (let assignee of users) {
let githubUser = settings.usermap[assignee as string];
if (githubUser) {
githubUser = '@' + githubUser;
} else {
githubUser = assignee as string;
}
organizationUsers.push(githubUser);
}

if (organizationUsers.length > 0) {
return `\n\n**${prefix}:** ` + organizationUsers.join(', ');
}

return '';
}