Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
2
2023-191
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Analytics
Analytics
CI / CD
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
2023-191
2023-191
Commits
3287c25c
Commit
3287c25c
authored
Aug 20, 2023
by
Tharuka Dilshan
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
New Api
parent
c896b352
Changes
5
Hide whitespace changes
Inline
Side-by-side
Showing
5 changed files
with
221 additions
and
0 deletions
+221
-0
Frontend/src/pages/AudioToSign.jsx
Frontend/src/pages/AudioToSign.jsx
+153
-0
SpringBootBackend/src/main/java/shrasthra/backend/controller/ShrasthraBackendController.java
...asthra/backend/controller/ShrasthraBackendController.java
+7
-0
SpringBootBackend/src/main/java/shrasthra/backend/service/i/ShrasthraBackendService.java
.../shrasthra/backend/service/i/ShrasthraBackendService.java
+2
-0
SpringBootBackend/src/main/java/shrasthra/backend/service/impl/ShrasthraBackendServiceImpl.java
...hra/backend/service/impl/ShrasthraBackendServiceImpl.java
+57
-0
SpringBootBackend/src/main/java/shrasthra/backend/util/common/REST_CONTROLLER_URL.java
...va/shrasthra/backend/util/common/REST_CONTROLLER_URL.java
+2
-0
No files found.
Frontend/src/pages/AudioToSign.jsx
0 → 100644
View file @
3287c25c
import
React
,
{
useState
}
from
'
react
'
;
import
axios
from
'
axios
'
;
import
{
useRecordWebcam
}
from
'
react-record-webcam
'
;
import
RestService
from
"
../services/RestService
"
;
import
'
../styles/audioToSign.css
'
;
const
AudioToSign
=
()
=>
{
const
[
isLogged
,
setIsLogged
]
=
useState
(
sessionStorage
.
getItem
(
"
isLogged
"
));
const
[
token
,
setToken
]
=
useState
(
sessionStorage
.
getItem
(
"
token
"
));
const
[
username
,
setUserName
]
=
useState
(
sessionStorage
.
getItem
(
"
username
"
));
const
[
userId
,
setUserId
]
=
useState
(
sessionStorage
.
getItem
(
"
userId
"
));
const
[
file
,
setFile
]
=
useState
(
null
);
const
handleFileChange
=
(
event
)
=>
{
const
selectedFile
=
event
.
target
.
files
[
0
];
setFile
(
selectedFile
);
};
// const handleUpload = () => {
// if (file) {
// try {
// const formData = new FormData();
// formData.append('audio1', file);
// axios.post('http://127.0.0.1:5000/uploadaudio', formData, {
// //axios.get('http://localhost:9000/', formData, {
// headers: {
// 'Content-Type': 'multipart/form-data',
// },
// });
// console.log('File uploaded successfully!');
// } catch (error) {
// console.error('Error uploading file:', error);
// }
// }
// };
//*************************************************** */
const
[
videoFile
,
setVideoFile
]
=
useState
(
null
);
const
[
audioFile
,
setAudioFile
]
=
useState
([]);
const
handleVideoFileChange
=
(
event
)
=>
{
setVideoFile
(
event
.
target
.
files
[
0
]);
};
const
handleAudioFileChange
=
(
event
)
=>
{
setAudioFile
(
event
.
target
.
files
[
0
]);
};
const
handleVideoUpload
=
()
=>
{
if
(
videoFile
)
{
const
formData
=
new
FormData
();
formData
.
append
(
'
video
'
,
videoFile
);
axios
.
post
(
'
http://127.0.0.1:5000/transcribe-audio
'
,
formData
)
.
then
(
response
=>
{
console
.
log
(
'
Video uploaded successfully
'
);
})
.
catch
(
error
=>
{
console
.
error
(
'
Error:
'
,
error
);
});
}
};
const
handleAudioUpload
=
()
=>
{
console
.
log
(
audioFile
)
const
fileData
=
new
FormData
();
// formData.append('audio1', audioFile);
fileData
.
append
(
'
file
'
,
audioFile
);
RestService
.
audioToSing
(
token
,
fileData
).
then
((
res
)
=>
{
// setMeaning(res.data.signMap.value);
console
.
log
(
res
);
}).
catch
((
err
)
=>
{
console
.
log
(
err
);
})
// axios.post('http://127.0.0.1:5000/transcribe-audio', formData)
// .then(response => {
// console.log(response);
// console.log('Audio uploaded successfully');
// })
// .catch(error => {
// console.error('Error:', error);
// });
};
return
(
<
div
className=
"col col-lg-12"
>
<
div
className=
'row'
>
<
h1
>
Upload Video
</
h1
>
</
div
>
<
div
className=
'row'
>
<
input
type=
"file"
accept=
"video/*"
onChange=
{
handleVideoFileChange
}
/>
{
/* <button onClick={handleVideoUpload}>Upload Video</button> */
}
<
button
className=
"btn video-btn"
onClick=
{
handleVideoUpload
}
>
Upload Video
</
button
>
</
div
>
<
div
className=
'row'
>
<
h1
>
Upload Audio
</
h1
>
</
div
>
<
div
className=
'row'
>
<
input
type=
"file"
onChange=
{
(
e
)
=>
setAudioFile
(
e
.
target
.
files
[
0
])
}
/>
{
/* <button onClick={handleAudioUpload}>Upload Audio</button> */
}
<
button
className=
"btn video-btn"
onClick=
{
handleAudioUpload
}
>
Upload Audio
</
button
>
</
div
>
<
br
/>
<
div
className=
"row"
>
<
div
className=
"col-6 col-sm-4"
>
<
h3
className=
"mt-4"
style=
{
{
fontFamily
:
'
FMAbayaBld
'
}
}
>
w¾:h (
</
h3
>
<
h1
className=
"d-flex align-items-center justify-content-center"
>
ffffs
</
h1
>
</
div
>
<
div
class=
"col-6 col-sm-4"
>
<
video
autoPlay
muted
className=
"video-box1"
/>
</
div
>
</
div
>
</
div
>
);
};
export
default
AudioToSign
;
SpringBootBackend/src/main/java/shrasthra/backend/controller/ShrasthraBackendController.java
View file @
3287c25c
...
...
@@ -45,6 +45,13 @@ public class ShrasthraBackendController {
return
responseDto
;
}
@RequestMapping
(
value
=
REST_CONTROLLER_URL
.
AUDIO_TO_SIGN
,
method
=
RequestMethod
.
POST
)
public
@ResponseBody
ProcessResponseDto
audioToSign
(
@RequestParam
(
"file"
)
MultipartFile
file
,
HttpServletRequest
request
){
ProcessResponseDto
responseDto
=
new
ProcessResponseDto
();
responseDto
=
backendService
.
audioToSign
(
file
);
return
responseDto
;
}
...
...
SpringBootBackend/src/main/java/shrasthra/backend/service/i/ShrasthraBackendService.java
View file @
3287c25c
...
...
@@ -8,5 +8,7 @@ import shrasthra.backend.dao.dto.ProcessResponseDto;
public
interface
ShrasthraBackendService
{
public
ProcessResponseDto
detectDynamicSign
(
MultipartFile
file
);
public
ProcessResponseDto
audioToSign
(
MultipartFile
file
);
}
SpringBootBackend/src/main/java/shrasthra/backend/service/impl/ShrasthraBackendServiceImpl.java
View file @
3287c25c
...
...
@@ -101,4 +101,61 @@ public class ShrasthraBackendServiceImpl implements ShrasthraBackendService{
return
responseDto
;
}
public
ProcessResponseDto
audioToSign
(
MultipartFile
file
)
{
ProcessResponseDto
responseDto
=
new
ProcessResponseDto
();
try
{
if
(!
file
.
isEmpty
())
{
// Convert the file to bytes
byte
[]
fileBytes
=
file
.
getBytes
();
RestTemplate
restTemplate
=
new
RestTemplate
();
// Set the URL of your Python Flask API
String
url
=
"http://127.0.0.1:5000/transcribe-audio"
;
// Create headers and set content type as multipart/form-data
HttpHeaders
headers
=
new
HttpHeaders
();
headers
.
setContentType
(
MediaType
.
MULTIPART_FORM_DATA
);
// Create the request body with the file as a part
MultiValueMap
<
String
,
Object
>
requestBody
=
new
LinkedMultiValueMap
<>();
requestBody
.
add
(
"fileData"
,
new
ByteArrayResource
(
fileBytes
)
{
@Override
public
String
getFilename
()
{
return
file
.
getOriginalFilename
();
}
});
// Create the request entity with headers and body
HttpEntity
<
MultiValueMap
<
String
,
Object
>>
requestEntity
=
new
HttpEntity
<>(
requestBody
,
headers
);
// Make the POST request to the second API
ResponseEntity
<
ProcessResponseDto
>
response
=
restTemplate
.
exchange
(
url
,
HttpMethod
.
POST
,
requestEntity
,
ProcessResponseDto
.
class
);
// Retrieve the response body
ProcessResponseDto
responseBody
=
response
.
getBody
();
// Print the response
if
(
responseBody
.
getSign
()
!=
null
)
{
SignMap
signMap
=
signMapRepository
.
findSignMapByLabel
(
responseBody
.
getSign
());
if
(
signMap
!=
null
)
{
responseDto
.
setSignMap
(
signMap
);
responseDto
.
setSuccess
(
true
);
}
}
}
}
catch
(
Exception
e
)
{
responseDto
.
setSuccess
(
false
);
System
.
out
.
println
(
"detectDynamicSign : "
+
e
);
}
return
responseDto
;
}
}
SpringBootBackend/src/main/java/shrasthra/backend/util/common/REST_CONTROLLER_URL.java
View file @
3287c25c
...
...
@@ -7,4 +7,6 @@ public class REST_CONTROLLER_URL {
public
static
final
String
DETECT_DYNAMIC_SIGN
=
"/sign/detectDynamicSign"
;
public
static
final
String
AUDIO_TO_SIGN
=
"/sign/audioToSign"
;
}
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment